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Preface 



Visual Prosthetics as a Multidisciplinary Challenge 

This is a book about the quest to realize a dream: the dream of restoring sight to the 
blind. A dream that may have been with humanity much longer than the idea that 
disabilities can be treated through technology - which itself is probably a very old 
idea. Long ago, when blindness was still considered a curse from the gods, some- 
one must have had the inspiration of building a wooden leg to replace one that had 
been crushed in a natural calamity or in battle. Many centuries lie between the 
concept of creating such a crude prosthesis to treat disability and today's endeavors 
to replace increasingly complex bodily functions, but the wish to restore useful 
function and the researchers' creative spirit remain the same. 

Around 1980, the developers of the cochlear implant were performing the first 
modest clinical trials of a technology to make the deaf hear again, or even hear for 
the first time. From those humble first attempts sprang a field that has become a 
model for modern neuroprosthetics, with tens of thousands of cochlear implants 
used successfully around the world. The development of the cochlear prosthesis 
illustrates the importance of bringing together professionals from a wide range of 
disciplines, from basic biology and engineering to rehabilitation, to create a func- 
tional substitute for a human sensory organ. 

In 1995, the editor of IEEE Spectrum magazine determined that artificial vision 
might be the next technological frontier, and that it should be the topic of a special 
issue. He invited a half dozen vision researchers to contribute articles about their 
expectations in two areas, visual prosthetics and machine vision, combined under 
the title "Towards an Artificial Eye." He instructed the authors not to feel con- 
strained by existing technology, but rather to envision the steps that would be 
required to replace natural vision. Most of the ideas presented in that May 1996 
issue have not yet been realized, especially those for prosthetic vision. Machine 
vision has made larger strides, which just goes to show that biology is more stub- 
born than technology - but also more resourceful, as machine vision researchers 
realize on a daily basis: Segmenting and recognition tasks that our visual system 
performs effortlessly can pose formidable problems for a computer-based image 
analysis system. Yet, encouragingly, some visual prosthesis designs predicted in 
that 1996 magazine are now being tested in clinical trials. 
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This is an exciting time for the field of visual prosthetics. Obviously, it is exciting 
for the hope it brings that vision can be restored. It is exciting for its challenge to 
researchers, technicians, clinicians, rehabilitation workers, and people in many 
other fields to commit their talents to the solution of a problem with so many 
dimensions. It is exciting for the experimenters when, seemingly against all odds, 
a blind study participant with a few dozen electrodes on the retina recognizes an 
object or letter "E" and finds a path around traffic cones in the lab without a cane 
or guidance. It is exciting for the participants in these trials, who feel they can play 
an active role in realizing the dream. It is exciting for their loved ones and the pub- 
lic at large, for whom the developments can't come quickly enough. And it is, 
unfortunately, too exciting for some media types who can't stop themselves from 
running ahead of the facts. 

This is also a field of setbacks, as when the new electrode coating that was sup- 
posed to withstand conditions inside the body for 20 years starts peeling off during 
its initial high-temperature soak test; of unpleasant surprises, as when the simple 
idea of putting together many small phosphenes to create an image runs up against 
the reality that phosphenes overlap and blur the image beyond recognition; and of 
patience put to the test, as when investors and the public do not get the miracle cure 
they may have been expecting. 

But mostly this is a field of great dedication by hundreds of researchers in doz- 
ens of labs in countries on four continents; of amazing tenacity by study partici- 
pants learning to make sense of a way of seeing that is so different from the vision 
they lost; and of true collegial spirit among all who share the dream, despite the 
realities of commercial interest. This collegial spirit was evident even in the days 
of the IEEE Spectrum issue: Throughout the 1990s, the National Institute of 
Neurological Disorders and Stroke sponsored an annual neural prosthesis workshop 
that was attended by all researchers competing for the scarce development funds 
then available for neuroprosthetics. Although the competition could be fierce, the 
annual workshop attendees formed a community that collectively solved stubborn 
problems of interfacing technology and biology, and attracted many new and tal- 
ented researchers to the field. Looking back, I feel that these workshops had a limi- 
tation: They were, by the nature of the research contracts given out, strongly geared 
towards technology, and less towards integration with physiology or rehabilitation. 
This was inherent in NINDS's mission to foster development of devices with broad 
application, but non-engineers were less likely to attend these highly technical 
gatherings. 

In the year 2000, Dr. Philip Hesburg at the Detroit Institute of Ophthalmology 
had the inspiration to foster a new collaboration among visual prosthesis research- 
ers, clinicians, and workers in low vision rehabilitation by creating and sponsoring 
a series of biennial meetings that he calls "The Eye and the Chip." Successful 
beyond Dr. Hesburg's expectations, these meetings have become the premier gath- 
ering place for researchers from all parts of the world and from very different 
backgrounds. Invited speakers are scientists who are advancing the field, yet the 
scale and atmosphere allow all researchers, patients, and the media to come and be 
updated about progress over the past 2 years. More perhaps than at other scientific 
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meetings, where investigators tend to gather within disciplines, participants at The 
Eye and the Chip are challenged to be open-minded, learn about and critique each 
other's work, and return home with fresh ideas for interdisciplinary approaches. 
The interdisciplinary character of this book reflects that same spirit. 

This book is also a reality check, an assessment of where we stand in 2010, 
almost 50 years after G.S. Brindley put the first revolutionary electrode assemblies 
under a blind patient's skull, yet in a field that is still very young. And this book is 
an introduction for people outside the field who may want to join the quest, or just 
be better informed. The book is unusual in being aimed at a readership as diverse 
as the disciplines contributing to the field: basic scientists, tissue and biomedical 
engineers, clinical researchers, and rehabilitation specialists. 

Most of all, this book is a tribute to the visionaries, the inventors, the creators of 
devices, the biomedical engineers, the surgeons and medical staff, the research 
psychophysicists, the occupational therapists, and the patient pioneers and their 
loved ones. In the chapters that follow, a few dozen workers in the field present their 
work and that of many colleagues. Each of their accounts conveys a passion for this 
multidisciplinary journey of discovery, a sense of urgency, a precise and meticulous 
effort to get it right and to learn - from the damaged visual system and from study 
participants - how to further improve the technology. 

If the reader comes away from this book with a sense of the breadth of the enter- 
prise, the hope for solutions that will truly help blind individuals, and the excite- 
ment shared by so many working in the field, then it has accomplished much of 
what the authors set out to do. If it allows practitioners in one discipline participat- 
ing in this development to get a better appreciation for what their colleagues in 
other disciplines are trying to accomplish, then the authors have clearly hit the right 
notes. And if it inspires enthusiastic young minds to join the quest, and to help turn 
the visual prosthesis into the next cochlear implant, then we will truly have 
succeeded. 

Baltimore, MD Gislin Dagnelie 

September 2010 
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Parti 

Structure and Function 

of the Visual System 



Chapter 1 

The Human Visual System: 

An Engineering Perspective 



Gislin Dagnelie 



Abstract This chapter provides a brief introduction to the architecture and function 
of the healthy visual system. Particular emphasis is placed on the diverse capabilities 
of the visual system that visual prosthesis researchers may want to emulate, to 
provide the reader with a realistic sense of the daunting challenges facing workers 
in this field. 
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4 G. Dagnelie 

1.1 The Visual System as an Engineering Compromise 

The purpose of this chapter is to outline the architecture and properties of the 
human visual system, but only to the extent required for a better understanding of 
its role as a substrate for visual prostheses. By sketching the properties of the 
healthy human visual system, we intend to provide the reader with an appreciation 
of the challenges one encounters in trying to reconstruct vision to the blind, even to 
a very modest level. Readers interested in more detailed or specific information 
regarding the visual system in health and disease are referred to some of the many 
excellent reviews in this area [12, 21, 22, 24, 49] and specifically to Chaps. 2-5 in 
this volume. 

Evolution of the vertebrate visual system over several hundred million years has 
provided the human eye and higher visual processing centers with ingenious 
compromises to allow sharp central vision, a wide field of view, color perception, 
and an enormous range of light-to-dark adaptation. Note the following benchmarks, 
unparalleled by any single man-made system: 

• The optic nerve, connecting the eye to the visual centers of the midbrain, has only 
approximately 1.2 million fibers [37] to represent the entire visual field (over 
140° horizontally and 120° vertically, or roughly 3.6 x 10 7 arcmin 2 ), in full color; 
a digital color camera with similar output bandwidth would provide 333,000 
pixels, i.e., about 630 pixels across the field or 13.3arcmin resolution. Yet the 
human eye achieves 1 arcmin resolution in the center of the visual field by 
combining variable cone photoreceptor spacing - from 0.4 arcmin (i.e., l/150th 
of a degree, or 0.0067 urn; foveola) to 3 arcmin (far periphery)[l, 2, 17, 38] - 
with variable post-receptoral convergence - from (on average) three ganglion 
cells per cone in the fovea to one ganglion cell per 6 cones in the far periphery 
[16, 56]. 

• The three color filters used in digital cameras have narrower bandwidths and 
wider color separation than the three human cone types. Yet the post-receptoral 
interactions in human vision allow discrimination over a wider range of color 
space than can be physically created with common light sources and pigments 
(pp. 306 ff. in [66]). 

• Both traditional cameras and the human eye employ mechanical apertures to 
adjust to a limited range of light levels (over 100 to 1 in cameras, about 15 to 1 
for the human pupil). This, however, represents only a fraction of the dark 
adaptation range required by changes in natural lighting conditions. Low noise 
properties of CCD chips and a variety of automatic gain control mechanisms 
allow modern cameras to function over a brightness range from less than 1 to 
over 100,000 lux. Rod photoreceptors in the human eye, however, extend the 
downward range by at least a factor of 1,000 [38], while cone dynamics extend 
the upward range by at least a factor of 10. The rod system transmits its infor- 
mation to the brain using the same optic nerve fibers used by the cones. 

• Compared to man-made detection and shape-recognition systems, the human visual 
system works quickly and with great precision: Attentional shift mechanisms 
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linked to movement and change in our peripheral vision lead to rapid redirection of 
gaze, in order to perceive detail in this novel stimulus [48]. Depth information is 
acquired monocularly by relative movement and size of objects in foreground and 
background [32, 47], while cooperative imaging by the two eyes, with disparities 
as small as a few arcsec (i.e., a small fraction of the width of a foveal cone), provides 
detailed depth and three dimensional shape information for nearby objects [63]. 
• Continuous information updating through rapid involuntary eye movements 
(microsaccades) is built into the human visual system, which can retain an accu- 
rate image only through such frequent "refreshment:" The perception of an 
image stabilized on the retina would fade after a few seconds [4, 27]. Man-made 
image acquisition systems may not spontaneously lose image information over 
time, but they require a recording medium if information is to be retained. 

Probably the most remarkable design accomplishment in the visual system is the 
fovea, or yellow spot, in the center of the retina: It combines the quality of the eye's 
optics close to the main axis, the density of photoreceptor cells in the central retina, 
the outward displacement of secondary neural elements and blood vessels from the 
retinal center [1, 68], and the ingenuity of retinal and cortical connectivity and 
processing to achieve the highest possible image resolution within the restrictions of 
limited anatomic resources and physiologic bandwidth set by a biological system. 

One of the greatest challenges in designing a visual prosthesis is, therefore, to 
reproduce the principal properties lost by eye disease or abnormal development, 
while using the capabilities provided by the remaining visual system to the greatest 
possible advantage. In order to gain a perspective of what visual prosthetics can and 
cannot accomplish for their recipients, it is important to understand how normal 
visual function depends on the anatomy and physiology of visual system compo- 
nent: the eye's optics, the retina, the pathways leading from retina to visual cortex, 
and the cortical areas involved in visual perception. A limited understanding of the 
operation and role of eye movements and binocular vision is also required. Some 
of these topics are briefly covered here; more detail can be found in Chap. 2. 



1.2 An Overview of Human Visual System Architecture 
1.2.1 Architecture and Basic Function of the Eye 

The structure and function of the eye correspond to the properties of visible light: 
Its optics form an image of the outside world on a photosensitive layer of cells, and 
the spectral properties of these cells construct multiple representations in different 
color bands. Specifically, the cornea, iris and crystalline lens (see Fig. 1.1), aided if 
necessary by corrective optics such as spectacles or contact lenses, cooperate to 
form a focused image on the back wall of the eye. The retina, a thin film covering 
the posterior half of the interior eye wall, contains multiple cell populations capturing 
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Fig. 1.1 Cross section through the human eye, showing the principal structures referred to in the 
text. Reprinted from [19J, with permission 



the image, pre-processing it for efficient information compression, and encoding it 
for transmission via the optic nerve to the visual centers in the brain. 

Figure 1.1 shows a horizontal cross-section of the adult human eye, including the 
optical elements, the retina, and the optic nerve. Light entering the eye is refracted 
by the cornea and the crystalline lens, and aperture-limited by the iris. The lens is 
suspended in a ring of thin fibers (zonula). The ciliary muscle behind the iris through 
the connecting zonula, can adjust the refractive power of the lens to maintain a sharp 
image when objects are brought closer to the eye; a membrane called the capsular 
bag surrounds the lens and separates the anterior and posterior chambers. Light passes 
unhindered through the watery fluid (aqueous humor) in the anterior chamber 
between cornea and lens, through the gel-like fluid (vitreous body) in the posterior 
chamber, and through the inner retinal layers, to reach the photoreceptors - the 
light-sensitive cells that convert it into electrical and chemical signals, initiating the 
process of vision. 

An intricate system of four straight and two oblique extraocular muscles allows 
the eye to be rapidly directed towards a visual target without requiring a head move- 
ment. More importantly, these muscles also allow for vergence (directing the two 
eyes to a common point at varying distances) and limited cyclo-rotation to counteract 
small rotations of the head or the scene and maintain a stable view of the world. 

The photoreceptors are situated in the deepest retinal layer and they, along with 
other cells in the outer retinal layers, receive nutrients and oxygen via a network of 
small capillaries under the retina, the choroid plexus. The inner retinal layers have 
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their own blood supply, which is fed through blood vessels in the optic nerve head; 
the arteries and veins of the inner retinal blood supply form two semi-circular 
patterns (the lower of these so-called arcades can be seen in Fig. 5.3, Chap. 5) 
around the central retina (or macula), which therefore has only narrow capillaries 
to limit interference with light projected onto the photoreceptors. The center-most 
portion of the macula (the fovea) does not contain any blood vessels and is called 
the foveal avascular zone. No inner retinal blood supply is required in this area, due 
to the outward displacement of all inner retinal cells, away from the center [1, 68]. 
Note the slight indentation of the retina at this so-called foveal pit, where the ability 
to capture details in the image is greatest. Also note the cupping of the retinal 
surface at the optic nerve (the optic nerve head; also called physiological blind spot, 
as it contains no photoreceptors), allowing for optic nerve fibers - actually the 
axons of retinal ganglion cells - to converge and enter the supporting structure of 
the optic nerve. 

To gain surgical access to the interior of the eye, one can make an incision in or 
near the cornea to enter the anterior segment, or cut the sclera (white outer wall of 
the eye) through the so-called pars plana, i.e., posterior to the attachment of the lens 
capsule, but anterior to the ora serrata, the forward edge of the retina. Retinal 
surgeons routinely use this latter route of access, and both the inner and outer 
retinal layers can be reached this way, albeit that reaching the outer retina requires 
an incision through the full thickness of the retina and the creation of an artificial 
detachment of the retina from the underlying retinal pigment epithelium (RPE) 
layer. In a healthy eye, re-attachment occurs naturally by resorption of subretinal 
fluid through the RPE. Recently, surgeons have also gained access to the outer 
retina by entering through the sclera behind the equator [71]. 

The retina forms a layered structure against the back wall of the eye, with 
photoreceptors (rods and cones) capturing the light; bipolar and ganglion cells 
passing the visual signal on towards the optic nerve, and horizontal and amacrine 
cells providing lateral interactions among cells in neighboring locations. Chapter 2 
provides greater detail regarding the different cell types in each retinal layer and 
their functions; here we will limit ourselves to the major structures that allow 
visual function to occur. 

In the normal retina, a highly structured arrangement of cells is seen in each 
layer. Under the retina, a layer of retinal pigment epithelium (RPE) cells fulfills the 
roles necessary to sustain the metabolism of the photoreceptors: The metabolic 
level of the photoreceptor outer segments is among the highest in the human body 
[36]. RPE cells supply nutrients and oxygen, regenerate phototransduction 
products, and digest debris shed by the photoreceptors [10]. 

Photoreceptors, the cells capturing the light, come in two main classes: rods, 
whose high internal gain allows vision at very low light levels [67], and cones, in 
short, medium, and long wavelength-sensitive types to allow color perception [25]. 
In both classes of cells the actual light capture and conversion takes place in the 
outer segment - indicated for the foveal cones in Fig. 1.2 by the abbreviation 
"COS" while the cell's inner segment, situated in the outer nuclear layer (ONL), 
provides the transduction to secondary neurons and regulates cell function. 
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Fig. 1.2 Cross section through the human fovea, showing the dense packing of elongated cone outer 
segments and the absence of the inner retinal layers across the "foveal pit." COS cone outer seg- 
ments; ONL outer nuclear laye; HL Henle fiber layer; INL inner nuclear layer. The Henle fibers 
connect foveal cones with the outwardly displaced bipolar cells in the INL. Reprinted from [57], 
with permission 



The distribution and packing of rods and cones varies dramatically across the 
retina. In the foveola, only medium and long wavelength-sensitive cones are found. 
In the surrounding foveal area, where the width of individual cones increases, and 
their packing density decreases accordingly, short wavelength-sensitive cones are 
also found, while rods are found only beyond the fovea. Figure 1.3, created 75 years 
ago on the basis of anatomical studies of donor retinas, still provides a fair repre- 
sentation of the density distribution of rods and cones along a horizontal line 
through the retina of a left eye. One may note that rod densities are highest around 
20° eccentricity. Cones are distributed throughout the entire retina, in roughly 
constant density beyond the central macula. Due to the decreasing convergence 
from photoreceptors to bipolar and ganglion cells in the inner retina, the visual 
acuity of both day and night vision gradually diminishes towards the periphery. 



1.2.2 Layout of the Retino-Cortical Pathway 



The connection between the eye and the central nervous system is formed by 
the fibers of the optic nerve. As noted above, these fibers, whose diameter is on the 
order of 1 urn, are the axons of retinal ganglion cells. Inside the eye, the fibers run 
along the inner retinal surface towards the optic nerve head in a characteristic 
pattern, such that fibers of the upper and lower retinal halves remain separated, and 
fibers close to the horizontal meridian, but far from the nerve head, arc away from 
this line to allow room for fibers originating closer to the nerve head. This orderly 
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Fig. 1.3 Horizontal cross section through the human retina, showing the rod and cone packing 
densities in the normal human retina. Note the very narrow area of high cone density, the highest 
rod density near 10° eccentricity, and the absence of photoreceptors in the physiological blind 
spot. Originally in [45]; this version from [39], with permission 



arrangement causes the fibers from the foveal area (which form 15 to 20% of all 
nerve fibers) to be located in the temporal quadrant of the optic nerve, at least for 
the anterior portion of its trajectory [27]. 

Once the axons enter the optic nerve, each fiber is encapsulated by a myelin 
sheath, formed by a class of cells called astrocytes; this sheath decreases the 
membrane conductance of the axons, increasing the conduction velocity and the 
length over which impulses can be conducted without severe attenuation [51]. 
Only at the so-called Ranvier nodes is the myelin sheath interrupted, allowing the 
impulses to be reinforced by virtue of the ion-gating properties of the local membrane. 

A cross-section through the human visual pathways can be seen in Fig. 1 .4. One 
may note that the predominant pathway leads from the eye to the lateral geniculate 
nucleus (LGN) of the thalamus, and from there to the occipital part of the cortex, 
while smaller numbers of fibers branch off to a tectal area, the superior colliculus, 
and to a number of pre-tectal nuclei. We will briefly discuss these subcortical path- 
ways below. 

Note also that the LGN and cortical areas exist in duplicate in the two halves of 
the brain. Each deals with one half of the visual world: The optic nerves from the 
two eyes meet in a structure called the optic chiasm, where fibers from the two 
nasal retinas cross over to combine with those from the temporal retina of the 
fellow eye; consequently each LGN and cortical hemisphere receive visual infor- 
mation from two corresponding retinal halves on their own side, and thus from the 
contralateral half of the visual field. 
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Fig. 1.4 Structure and location of the human primary visual pathways, in relation to other major 
brain structures. The left cerebral hemisphere, with the exception of the occipital cortex, has been 
removed; the left LGN is hidden by the optic radiations (arrow). Reprinted from [19], with 
permission 



The LGN has a layered structure, with pairs of layers receiving axons of different 
ganglion cell types, and each layer in a pair receiving signals from one eye. 
Interactions between layers in the form of overall suppression when the retina in 
the fellow eye is stimulated layers have been demonstrated [53], but localized inter- 
actions across layers do not occur; this indicates that binocular processing required 
for stereopsis does not take place until the level of the visual cortex. The gateway 
function of the LGN, which in other mammals such as the cat appears to play a 
crucial role in adaptation and attention, and through which signals from the two 
eyes can mutually inhibit each other [29], is thought to be less prominent in 
primates, including humans. Yet anatomical feedback connections from a number 
of subcortical nuclei onto the LGN are as extensive in monkey as in cat [8], and 
gating functions related to circadian rhythms and other systemic conditions are 
therefore plausible in primates as well. 

Forward pathways from the LGN lead to the primary visual cortex (VI, also 
called striate cortex; these fibers form the optic radiation), but also to higher visual 
cortical areas and to subcortical areas such as the superior colliculus (SC). The role 
of the extrastriate cortical pathways is still a topic of speculation and investigation; 
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from clinical cases it is evident, however, that patients with lesions to the striate 
cortex acquired after childhood retain little or no useful vision [11]. The roles of the 
tectal pathways, including mutual connections between cortical areas, the SC and 
the pulvinar, are also subject to active research. It has been found, for example, that 
cortical connections with midbrain areas are essential for maintaining and shifting 
attention, rather than for processing detailed visual information [35]. 

The visual cortex occupies the occipital and parts of the parietal and temporal 
lobes of the cerebral cortex. Like the entire cortex, it forms a highly folded struc- 
ture, with a thickness of approximately 1.5 mm. It is surrounded by the cerebrospinal 
fluid, several layers of meninges - pia mater, arachnoid, and dura -, and the skull. 
Especially the skull forms an important barrier to any attempt at functional electrical 
stimulation of cortical cells. 

Like the retina, the visual cortex is a layered structure, in which different cell 
groups perform different tasks. Along its two-dimensional surface, one finds an 
orderly mapped representation of the outside world. Contrary to the retina, how- 
ever, the cortex consists of multiple areas, hierarchically organized, each of which 
performs a partial processing task in the analysis of the scene around us. At the 
present time, over 30 visual cortical areas per hemisphere are recognized in 
monkey, and a similar number of distinct areas is thought to exist in humans [61]. 

The first cortical representation, in the striate cortex, is shown schematically in 
Fig. 1.5. It presents a straightforward map of the visual world, but contains four 
major transformations: 

The projection from the LGN (and thus retinal ganglion cells) onto VI input 
cells has approximately constant density, which means that the central visual field 
is highly over-represented in the visual cortex: Roughly 20% of VI represents the 
retinal fovea, and thus the central 1-2° of the visual field, with rapid drop-off of the 
density towards the periphery. This inhomogeneous map is conveniently expressed 
by the cortical magnification factor, M('), i.e., the number of mm of cortex devoted 
to 1° of retina, as a function of eccentricity [20, 34]. 

The folding of the human cortex, prompted by the evolutionary expansion of 
higher (cognitive) processing, has resulted in an arrangement where most of the 
peripheral visual field is represented in portions of V 1 that are buried in the medial 
walls and sulci of the cerebral hemispheres. Only the foveal representation, situated 
along the border of VI and the adjacent area V2, is exposed at the surface of the 
occipital cortex, approximately 1 cm above the inion, a protruding portion near the 
bottom of the skull bone on the back of the head. More peripheral visual field areas 
are represented along the medial walls of the cerebral hemispheres, and in deep sulci 
embedded within these areas. 

The complex representation of the visual field onto area VI, combined with the 
lack of accessibility due to cortical folding, greatly reduces our ability to investigate 
and stimulate the peripheral visual field. Area V2 and several higher areas form the 
exposed portion of the occipito-parietal cortex, and would seem to provide better 
opportunities for peripheral field stimulation. However, while these areas may 
appear to be more easily accessible than VI, they have an equally dense pattern of 
sulci, which greatly vary between individuals; moreover, receptive field properties 
and visual field maps become increasingly complex in higher cortical areas [41]. 
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Fig. 1.5 VI projection of the visual field. The medial wall and part of the occipital surface of the 
left cerebral hemisphere (a) and the corresponding visual fields for the two eyes (b) are shown. 
Note that the projection of the fovea (F) and a narrow surrounding hemicircle of the visual field 
project onto the occipital cortex, with the projection of the vertical meridian adjacent to area V2, 
whereas more peripheral areas - including most of the macula - projects to the medial wall of the 
cortex, with much of the projection buried in the calcarine fissure (C). Also note that the left 
hemisphere receives information from the right visual hemifield, that the superior visual field 
projects to the inferior part of VI - i.e., gross localization is preserved from retina to VI, and that 
corresponding retinal locations in the two eyes project to the same cortical location. No matching 
locations exist for the far nasal segment of the right retina (60-90°), as the bridge of the nose 
blocks the corresponding area in the left eye. Reprinted from [19], with permission 
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1.2.3 Layout of the Subcortical Pathways 

In addition to the visual pathway to LGN and striate cortex, which receives the 
great majority of retinal ganglion cell axons, there also are subcortical pathways, 
formed by optic nerve fibers projecting to pretectal nuclei and to the pulvinar. 
In primates, the projections to the pregeniculate nucleus and pulvinar are thought 
to be of minor importance, and may be thought of as anatomical remnants: In lower 
mammals, ablation of striate cortex at birth allows these projections to greatly 
increase in density, leading to the development of crude functional vision, but similar 
experiments in newborn monkeys show neither the proliferation of projections nor 
appreciable acquisition of visual function [14, 15]. Other projections, however, in 
particular those to the pretectal nucleus of the optic tract (NOT) and the terminal 
nuclei (TN) of the accessory optic system, have been demonstrated to play an 
important role in the rapid control of eye position through vestibulo-ocular reflex, 
saccades, and sustained fixation [33, 43]. 

Detailed studies of anatomy and physiology of the primate eye movement 
system over the last several decades in awake, trained animal models, have 
shown that the NOT receives information on "retinal slip," i.e., generalized 
displacement of the retinal image [58]. This retinal slip signal is encoded as a 
velocity signal, and serves as input to the neural integrator in the nucleus prep- 
ositus hypoglossi [50]. Pathways between the NOT and primary visual cortex (as 
well as multiple similar projections between cortical and subcortical structures) 
are also known to exist, and have been shown to compensate in part for lesions 
to the NOT or its retinal input [23, 31]. 



1.3 An Overview of Human Visual Function 

The anatomy and physiology of the visual system presented above can help us 
understand many of the properties of normal vision, and some of the vision defects 
experienced by patients with blinding eye diseases. We will briefly discuss the 
aspects most pertinent in understanding the requirements for neural visual 
prostheses. 



1.3.1 Roles of Central (Foveal) Vision 

Central visual function is more than just the utilization of the denser packing of 
photoreceptors in the central retinal area and the higher density of ganglion cells 
per photoreceptor in this area. These properties of the retina would account for 
basic properties such as good two-point resolution, but they would not explain why 
foveal vision is superior to peripheral vision in many other ways. The following 
major areas of foveal specialization should be considered. 
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Spatial integration tasks, e.g., hyperacuity. Normally sighted observers have the ability 
to resolve small deviations in alignment of parallel or abutting lines, small angular dif- 
ferences and displacements, all on a scale well below 1 arcmin, the spacing of foveal 
photoreceptors. Such "hyperacuities" apparently rest on the ability of foveal projec- 
tions in cortical areas to combine the precise positional coding of earlier stages in the 
visual system over increasing distances, using feedback and tuning mechanisms that 
have been honed by years of experience. The notions of learning and tuning are sup- 
ported by the lack of hyperacuity in subjects with inherited abnormalities of foveal 
development and eye movements [64] or with developmental deficits [9], and by the 
gradual acquisition of hyperacuity performance throughout childhood [69]. 

Stereopsis. Combination of the signals from corresponding locations in the two 
retinas, in a highly systematic fashion, is required for perception of depth in stationary 
three-dimensional scenes. This function takes place at and beyond the VI cortical 
level [55]. Fusion of the two retinal images on an object of interest defines a curved 
plane, the horopter, formed by the collection of points being imaged at exactly 
corresponding locations on the retinas of the two eyes. Finely-tuned disparity neu- 
rons detect left-right eye correspondence of retinal locations for points slightly in 
front of (crossed disparity) or beyond (uncrossed disparity) the horopter, with resolution 
on the order of arc seconds, similar to that seen in hyperacuity task performance. 

Complex pattern recognition and discrimination tasks, e.g., face recognition. 
Beyond the ability to make precise visual judgments enabled by the high resolution 
of foveal vision, normally-sighted observers acquire great skill at memorizing, 
recognizing, and discriminating among patterns, varying from feature discrimina- 
tion in the natural environment, such as recognizing human faces, to the processing 
of complex man-made forms and objects, such as reading text or maps. These capa- 
bilities require both high-level visual processing skills and cognitive brain functions 
such as leaning and memory. It is not necessarily true that these specialized skills 
cannot be acquired in peripheral vision: Certainly, a person with a central scotoma 
(blind area) due to macular degeneration can read, if given text with enough 
magnification and contrast [26, 40]. Nonetheless, these skills appear to depend 
critically on specialization during early phases of development, and functions such 
as reading, that once were linked to foveal visual function, can only partially, and 
with great effort, be taken over by extrafoveal vision, as if the task of vision itself 
has to be re-learned [28]. On the other hand, children with poorly developed foveal 
vision, such as those with albinism or aniridia, can learn to proficiently read and 
recognize patterns or faces, if given adequate magnification, and the same intensive 
exposure as their normally-sighted peers [30]. 

Visuomotor integration tasks, e.g., handwriting. These tasks are very similar to the 
pattern recognition tasks described above, in that they require complex visual 
processing and memory functions, but moreover they require integration with 
proprioceptive and motor functions distributed across many different brain areas. 
Some of these tasks may depend less critically on foveal function, but inasmuch as 
they are based on skills learned during early development, their execution often 
proves difficult when foveal vision becomes impaired later in life [44]. 



1 The Human Visual System: An Engineering Perspective 1 5 

Note that all the skills referred to as specific for foveal vision involve the ability 
to see fine detail, combined with extensive learning throughout the critical period 
of development. 



1.3.2 Roles of Peripheral Vision 

The role of peripheral vision in performing daily activities is often underappreci- 
ated. Most human-designed visual tasks rely on the perception of detailed shapes, 
but there are important exceptions: Noticing traffic off to the side while driving 
and keeping track of fellow and opponent players during team sports require con- 
tinuous processing of events and objects throughout the visual field. Similarly, 
observing wildlife and other outdoor activities require the use of our entire visual field. 
In almost all cases, these visual tasks require us to perceive motion and other 
changes, and it should come as no surprise that the evolution of the vertebrate 
visual system has favored use of the periphery for precisely these functions [6]. On 
the other hand, our attention tends to be focused on objects and events in central 
vision, whereas school children with severely impaired central vision appear to use 
their peripheral vision much more efficiently than normally- sighted individuals [65]. 
One of the surprising aspects of peripheral vision is how much of it can be lost 
before a person becomes aware of the change. Thus disorders such as glaucoma and 
retinitis pigmentosa may go undetected well beyond the point where irreversible 
damage to cells in the peripheral retina has occurred [13, 42]. 



1.3.3 Roles of Dark-Adapted Vision 

As was mentioned above, cones are unable to function effectively at light levels 
below 0.003 cd/m 2 . At these low illumination levels, rod photoreceptors continue 
to be effective, by virtue of the high gain, multi-stage phototransduction cascade in 
the rod outer segment. On the other hand, at intensities above 3 cd/m 2 rod function 
is actively suppressed. Rods are not distributed evenly throughout the retina: The 
center of the retina, with a diameter of approximately 5°, forms a rod-free zone, and 
the highest density of secondary retinal cells receiving rod signals is situated 
between at eccentricities between 5 and 10°, as evidenced by the common experi- 
ence that a dim object at night is best observed by intentionally looking slightly 
away from the object. 

Dark-adapted vision differs from daytime vision in two important respects, 
both related to the need for maximum sensitivity, i.e., the detection of a very small 
stimulus signal in the ongoing background of visual noise (spontaneous activity of 
retinal cells). Ganglion cells in the dark-adapted retina integrate signals from a 
much larger number of photoreceptors than in the light-adapted retina [60], and 
the time course over which this integration takes place is significantly extended [7]. 
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For this reason it is not possible to see small or rapidly moving objects at very low 
light levels that allow only rod vision. 

In designing image capture systems for prosthetic vision, researchers may want 
to borrow some of the principles employed by the retina. Increased integration 
times are commonly employed in CCD arrays and other electronic camera detectors, 
but integration across space is rarely employed for real-time image acquisition, as 
it runs counter to the designers' wish to maximize spatial detail. Given the limited 
spatial resolution of early visual prostheses, however, such loss of spatial detail at 
the input should be of no consequence to the image perceived by the prosthesis 
wearer, and could be employed to achieve maximum sensitivity. 



1.3.4 A Few Remarks Regarding Visual Development 

Throughout this chapter we have seen that the principal pathway on which func- 
tional vision depends is that from the retina through the LGN to cortical area VI. 
Even the mechanisms of involuntary eye movement control (maintaining off-center 
gaze, microsaccades), which are served by pretectal pathways, can be compensated 
for - as appears from primate experiments -, presumably by virtue of cortico- 
subcortical connections like those from VI to the NOT and accessory optic nuclei. 
Hence, if visual impairment or blindness is caused by a disorder at the level of the 
eyes, optic nerve, or primary visual cortex, in a person whose visual function had 
followed its normal development earlier in life, one can assume that all cortical 
processing mechanisms and functions are intact, and may be successfully restored 
if adequate input signals are provided. If, on the other hand, normal visual develop- 
ment did not occur, as in the case of a congenital deficit of the retina or optic nerve, 
then a visual prosthesis implanted at a later age is unlikely to enable functional 
vision, similarly to the lack of functional vision documented in adult corneal trans- 
plant recipients who had congenital corneal opacities or corneal trauma in infancy 
[59]. Just like the cochlear prosthesis, however [54, 62], the visual prosthesis may 
provide opportunities for partial development of functional vision in children, 
provided the implantation takes place at a very young age, presumably in the first 
or second year of life. Obviously, this will require the technology to have been 
proven safe and effective in adults. 



1.4 Prospects for Prosthetic Vision Restoration 

On the basis of the architectural and functional layout of the human visual system 
described above, it should be clear that vision restoration through stimulation of 
intact structures along the retino-cortical pathway is feasible, in principle. Given 
the transformation of visual information that occurs at every stage along the visual 
pathway, the prudent approach in visual prosthetics would seem to be to implant as 



1 The Human Visual System: An Engineering Perspective 



17 




After: Polyak 



Fig. 1.6 In this cross-section of the human retina-cortical pathway, seen from below, the numbers 
1 through 4 indicate locations currently being considered for visual prosthesis implantation. 
Adapted from [46] 



distally - that is, as early along the pathway - as feasible: If the photoreceptors are 
non-functional, then the bipolar or ganglion cells in the retina would be the best 
target for stimulation; if the retina is detached so a retinal prosthesis cannot be placed 
reliably, then an optic nerve implant would be indicated; if the retinal ganglion cells, 
and thus the optic nerve fibers, are damaged by glaucoma, then an implant in the LGN 
or primary visual cortex may be in order; etc. Figure 1 .6 illustrates the four locations 
that seem best suited for the placement of visual prostheses. 

In all these examples the assumption is that the visual pathway proximal to the 
lesion - that is, towards the brain - is intact, but their success will depend on the 
extent of secondary degeneration that may have occurred further along the visual 
pathway. Certainly it is known that many ganglion cells are lost after an extended 
period of outer retinal degeneration, but a substantial percentage survives, more 
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than enough to carry the small number of signals from today's implants [52]. 
Moreover, the more central and the more invasive the surgery, the greater the risk of 
systemic and irreversible complications. Thus the idea to implant on the proximal 
side of the lesion, but as close to it as can be done safely, appears to be the implicit 
practice among most visual prosthesis groups. 

The extent to which prosthesis recipients will be able to regain useful vision, and 
the duration required for functional rehabilitation, cannot be predicted until a larger 
number of patients has received a greater variety of implants than is currently 
the case; recent reports from two groups regarding letter recognition [18, 70], 
wayfinding [3], and maze tracing [5] by a small number of retinal implant recipients 
are encouraging indicators that a modest level of prosthetic vision is possible. From 
simulations in sighted volunteers (see Chap. 16) we have learned that seeing with 
pixelized vision is possible; yet the small electrode numbers in retinal arrays, the 
irregularity of phosphenes in cortical arrays, and the apparent differences between 
simulations with distinct dots and prosthetic percepts of broadly overlapping phos- 
phenes will make the rehabilitation process an arduous one. 

Acknowledgment Supported in part by PHS grant # EYO 19991. This chapter is an adaptation of 
parts of an earlier chapter. [19] The author wishes to acknowledge the contributions of Eyal 
Margalit, M.D., who co-authored that chapter. 
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Chapter 2 

Vision's First Steps: Anatomy, Physiology, 

and Perception in the Retina, Lateral Geniculate 

Nucleus, and Early Visual Cortical Areas 

Xoana G. Troncoso, Stephen L. Macknik, and Susana Martinez- Conde 



Abstract This chapter reviews the functional anatomical bases of visual perception 
in the retina, the lateral geniculate nucleus (LGN) in the visual thalamus, the 
primary visual cortex (area VI, also called the striate cortex, and Brodmann area 17), 
and the extrastriate visual cortical areas of the dorsal and ventral pathways. 

The sections dedicated to the retina and LGN review the basic anatomical and 
laminar organization of these two areas, as well as their retinotopic organization 
and receptive field structure. We also describe the anatomical and functional differ- 
ences among the magnocellular, parvocelullar and koniocellular pathways. 

The section dedicated to area VI reviews the functional maps in this area (retino- 
topic map, ocular dominance map, orientation selectivity map), as well as their 
anatomical relationship to each other. Special attention is given to the modular 
columnar organization of area VI, and to the various receptive field classes in VI 
neurons. 

The section dedicated to extrastriate cortical visual areas describes the "where" 
and "what" pathways in the dorsal and ventral visual streams, and their respective 
physiological functions. 

The temporal dynamics of neurons throughout the visual pathway are critical to 
understanding visibility and neural information processing. We discuss the role of 
lateral inhibition circuits in processing spatiotemporal edges, corners, and in the 
temporal dynamics of vision. 

We also discuss the effects of eye movements on visual physiology and percep- 
tion in early visual areas. Our visual and oculomotor systems must achieve a very 
delicate balance: insufficient eye movements lead to adaptation and visual fading, 
whereas excessive motion of the eyes produces blurring and unstable vision during 
fixation. These issues are very important for neural prosthetics, in which electrodes 
are stabilized on the substrate. 
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Finally, another critical issue for neural prosthetics concerns the neural code for 
visual perception: How can the electrical activity of a neuron, or a neuronal popula- 
tion, encode and transmit visual information about an object? Here we will discuss 
how neurons of early visual areas may communicate information about the visible 
world to each other. 

Abbreviations 

area MST Medial superior temporal area 

area MT Middle temporal visual area 

area V 1 Primary visual cortex 

DOG Difference of gaussians 

GABA Gamma-aminobutyric acid 

LGN Lateral geniculate nucleus 



2.1 Introduction 

The process of "seeing" is complex and not well understood. But we do know that 
individual neurons in the early visual system are tuned to stimuli with specific 
attributes (such as color, shape, brightness, position on the retina, etc.). The recep- 
tive field of a visual neuron is the area of the visual field (or its corresponding 
region on the retina) that when stimulated (by light or electrical impulses) can influence 
the response of the neuron (Fig. 2.1). Visual stimuli outside a neuron's receptive field 




Fig. 2.1 Activation of retinal photoreceptors and their corresponding receptive fields during visual 
exploration. The eye focuses light that is reflected from the visual image onto the retina, upside 
down and backwards. Adjacent photoreceptors within the retina are activated by adjacent points of 
light from the painting. Figure by the Barrow Neurological Institute Illustrations Department 
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produce no effect on the neuron's responses. Understanding the precise receptive 
field structure of a given neuron is crucial to understanding and predicting its 
responses to specific stimuli. For instance, some early receptive fields have a spatial 
substructure (while others do not), and stimulating their different subregions results 
in increases or decreases in neural activity. A visual neural prosthesis should ultimately 
replace the visual processing represented by the receptive fields at a given 
(damaged or otherwise impaired) stage of the visual hierarchy. A close replication 
of the output of the replaced neurons will ensure that the healthy tissue farther 
along the visual pathway receives properly structured inputs. 



2.2 Retina 
2.2.1 Anatomy 

Vision starts in the retina: it is here where photons are converted into electrical signals, 
to be then interpreted by the brain to construct our perception of the visual world. 

The retina has the shape of a bowl (about 0.4 mm thick in adult humans). It is a 
well organized structure with three main layers (called the nuclear layers) of 
neuronal bodies. These main layers are separated by two other layers containing 
synapses made by axons and dendrites (called the plexiform layers). The basic 
retinal cell classes and their interconnections were revealed by Ramon y Cajal over 
a century ago [175] (Fig. 2.2). 




Outer piexiform 
layer 

Inner nuclear 

layer 

Inner plexiform 
layer 

Ganglion cell 
layer 



Fig. 2.2 Retinal layers, (a) Light micrograph of a vertical section of the human retina from [29]. 
(b) Cross-sectional microscopic drawing by Ramon y Cajal from [127, 176] 
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The functional anatomy of the retina is enormously rich and complicated. A short 
overview is provided here, to set a basis to understand the next few stages of the 
visual hierarchy. 

The three nuclear layers are the photoreceptor layer (which lies on the back on 
the retina, farthest from the light coming in), the inner nuclear cell layer (in the 
middle) and the ganglion cell layer (nearest to the center of the eye). 

Photoreceptor layer, light is transduced into electrical signals by photoreceptors: 
rods and cones. Cones are not sensitive to dim light, but under photopic conditions 
(bright light) they are responsible for fine detail and color vision. Rods are respon- 
sible for our vision under scotopic conditions (dim light), and saturate when the 
level of light is high. Rods and cones are distributed across the retina with very 
different profiles: in the fovea, where our fine vision is most detailed, cones are 
very densely packed (up to 160,000 cones/mm 2 ) but cone density drops rapidly as 
we move away from the fovea. Rods are absent from the fovea [190], but their 
density rises quickly to reach a peak at an eccentricity between 5 and 7 mm, beyond 
which they steadily decline in number [45, 46, 164]. Humans have one type of rod 
and three types of cones. The three types of cones, responsible for color vision, are 
called L (or red) cones, M (or green) cones, and S (or blue) cones, and they are most 
sensitive to different segments of the spectrum of light: L cones are most sensi- 
tive to long wavelengths (peak sensitivity at 564 nm), M cones are most sensitive 
to middle wavelengths (peak sensitivity at 533 nm) and S cones are most sensitive to 
short wavelengths (peak sensitivity at 437 nm) [32, 33, 131]. L, M, and S cones are 
distributed in the retina in a particular way: only 10% of the cones are S cones, and 
they are absent from the fovea. Although L cones and M cones are randomly inter- 
mixed, there are ~2 times more L cones than M cones [1, 41, 44, 152, 187]. 

Inner nuclear layer, contains three classes of neurons: horizontal cells, bipolar 
cells, and amacrine cells. Horizontal cells have their bodies in the inner nuclear 
layer and connect to photoreceptors (through chemical synapses) and other hori- 
zontal cells (through gap junctions) in the outer plexiform layer [223]. Horizontal 
cells receive input from photoreceptors, but they also give output to the same pho- 
toreceptors, providing lateral inhibition, which acts to enhance spatial differences 
in photoreceptor activation at the level of the bipolar cells [49, 222]. There are over 
13 different types of bipolar cells [30, 105] and all of them have some dendritic 
processes in the outer plexiform layer, the soma in the inner nuclear layer and some 
axon terminals in the inner plexiform layer [66]. The dendritic processes of a bipo- 
lar cell receive input from one type of photoreceptor (either from cones or from 
rods, but never from both) [186]. Each bipolar cell then conveys its response to the 
inner plexiform layer, where it contacts both amacrine and ganglion cells [49]. 
Amacrine cells (over 30 different types), receive input from bipolar cells and other 
amacrine cells, and pass their messages onto bipolar cells, other amacrine cells, and 
ganglion cells [50, 128]. Different types of amacrine cells may have different func- 
tions in retinal processing, but their specific roles remain unknown for the most part. 

Ganglion cell layer, there are more than 20 different ganglion cell types [105], and 
many of them are specialized on coding some particular aspect of the visual world 
such as sign-of-contrast and color [186]. Ganglion cells receive their input from 
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amacrine and bipolar cells, and send their outputs to the brain in the form of action 
potentials through the optic nerve. These are the first cells in the visual pathway that 
produce action potentials (all-or-none) as their output; all the previous cell classes 
(photoreceptors, horizontal, bipolar and amacrine cells) release their neurotrans- 
mitters in response to graded potentials. Even though there are over 20 different types 
of ganglion cells, two of them account for almost 80% of the ganglion cell population 
[171]: the midget and the parasol ganglion cells, named by Polyak [173]. Near the 
fovea each midget ganglion cell receives direct input from only one midget bipolar 
cell [104, 106] and thus has a very small and compact receptive field (it collects input 
from a small number of cones). Parasol cells receive their direct input from diffuse 
bipolar cells, have larger dendritic fields, and thus receive input from many more 
cones [224] . The dendritic field size increases with eccentricity for both types of cells 
[48, 51, 224]. Away from the fovea, the increase in dendritic field size with retinal 
eccentricity is more or less matched by a decrease in spatial density, so the amount of 
retina covered is approximately constant over most of the retina [224]. 



2.2.2 Physiology and Receptive Fields 

The receptive fields of ganglion cells in the retina are approximately circular and 
have functionally distinct central and peripheral regions (called center and sur- 
round); stimulation of these two regions produces opposite and antagonistic effects 
upon the activity of the ganglion cells. Ganglion cells respond optimally to differen- 
tial illumination of the receptive field center and surround. Diffuse illumination of 
the whole receptive field produces only weak responses. There are two main types 
of center-surround receptive fields: on-center receptive fields respond best to light 
falling on the center, and darkness falling on the surround; off-center receptive fields 
respond best to darkness on the center and light on the surround (Fig. 2.3). 
The properties of center-surround receptive fields change during scotopic conditions: 
the size of the receptive field center usually increases, the surround strength dimin- 
ishes and there is a longer latency for the response [16, 28, 65, 144, 156, 167]. 

Werblin and Dowling [225], and Kaneko [99] discovered that bipolar cells also 
have center-surround receptive fields. 



Fig. 2.3 Concentric recep- 
tive fields of retinal ganglion 
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In the dark, photoreceptors are depolarized and continuously active [205], 
releasing glutamate to bipolar and horizontal cells. When light arrives and the photo 
pigments bleach within a photoreceptor, that photoreceptor hyperpolarizes, and 
the amount of glutamate released decreases in a graded manner, as a function of the 
number of photons [204]. All photoreceptors use the same neurotransmitter, glutamate, 
and so on-center and off-center bipolar cells acquire their preference by having one 
of two types of glutamate receptor [150]: 

- On-center bipolar cells have metabotropic receptors that make the cell hyperpolarize 
when they receive glutamate [159, 199]. When light hits photoreceptors, they 
hyperpolarize and release less glutamate. This reduces the inhibition in the bipolar 
cells that therefore increase their activity. In the dark, photoreceptors depolarize 
and release more glutamate. Therefore the bipolar cells hyperpolarize. 

- Off-center bipolar cells have ionotropic receptors that depolarize the cell when 
receiving glutamate [161, 200]. In this case, when light arrives to the retina, 
the photoreceptors hyperpolarize and release less glutamate. Consequently, the 
bipolar cells decrease their activity. In the dark, the photoreceptors depolarize 
and release more glutamate. As a consequence, the bipolar cells depolarize. 

Both on- and off-bipolar cells make the same kind of contacts in the inner plexiform 
layer. All bipolar cells release glutamate as their neurotransmitter and all the ganglion 
cells have ionotropic receptors: therefore, ganglion cells that receive input from on- 
center bipolar cells are also on-center. Ganglion cells that receive input from off-center 
bipolar cells are off-center [186]. In 1978 Nelson et al. discovered that there is a clear 
anatomical difference between on- and off-bipolar cells: they synapse onto ganglion 
and amacrine cells within different sublayers within the inner plexiform layer. The 
off-center bipolar dendrites make synapses closer to the inner nuclear layer whereas 
the on-center bipolar dendrites terminate closer to the ganglion cell layer [47, 160]. 

As described earlier, there are two predominant types of ganglion cells: midget 
and parasol [173]. Both types of ganglion cells have center-surround receptive 
fields with similar spatial organization, but physiological studies have described 
several differences between them: parasol cells respond more transiently to light 
onset or offset than midget cells [82]; parasol cells have larger receptive fields cen- 
ters than midget cells at the same eccentricity [55]; most midget cells have spectral 
selectivity and antagonism while most parasol cells do not [55, 57, 82]; parasol 
cells respond much more vigorously than midget cells to small changes in lumi- 
nance contrast [102]. The anatomical and functional differences between midget 
and parasol cells lead to two different visual pathways that remain segregated 
throughout the early visual system. The parvocellular pathway starts with the 
midget cells and is very sensitive to color and spatial frequency. The magnocellular 
pathway starts with the parasol cells and is most sensitive to luminance contrast and 
temporal frequency. 

Due to the center-surround organization of the ganglion cell receptive fields, 
these neurons are quite insensitive to changes in overall levels of luminance. They 
signal differences within their receptive fields by comparing the degree of illumination 
between the center and the surround. 
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2.3 LGN 

All retinal ganglion cells send their axons to the brain via the optic nerve. The 
axons decussate at the optic chiasm, so the information from each nasal hemiret- 
ina is sent to the contralateral hemisphere. Retinal ganglion cells project to three 
major subcortical targets: the pretectum, the superior colliculus, and the lateral 
geniculate nucleus (LGN) of the thalamus. The LGN is the principal structure 
that sends visual information to the visual cortex, with input from 90% of the reti- 
nal ganglion cells. The LGN is laid out so that neighboring neurons are stimu- 
lated by adjacent regions in visual space. This property is called retinotopic 
organization. 

In primates, the LGN contains six layers of cell bodies that can be classified 
in two groups according to their histological characteristics: the two bottom layers 
(ventral) contain large cell bodies and are called magnocellular layers; cells in the 
four upper layers (dorsal) are smaller and are called the parvocellular layers. The 
parvocellular layers receive their main inputs from the midget ganglion cells in 
the retina. The magnocellular layers receive their main inputs from parasol gan- 
glion cells [42, 109, 170, 189, 192]. Between each of the magno and parvo layers 
lies a zone of very small cells: the koniocellular layers. Konio cells are function- 
ally and neurochemically distinct from magno and parvo cells [87]. The finest 
caliber retinal axons, presumably originating from retinal ganglion cells that are 
morphologically distinct from those projecting to magno and parvo layers [109], 
innervate the koniocellular layers [42]. The koniocellular pathway starts with the 
small bistratified ganglion cells of the retina that are sensitive to blue (or S-cone) 
activation. The koniocellular layers interdigitate between the primary six layers 
of the LGN [86]. 

Each LGN receives input from both eyes, but the input from each eye is seg- 
regated to different monocular layers: layers 1,3, and 6 get input from the con- 
tralateral eye, whereas layers 2, 4, and 5 get input from the ipsilateral eye [94]. 

Hubel and Wiesel discovered that LGN receptive fields have a similar center- 
surround configuration to retinal ganglion cells, however the suppressive strength 
of the surround is stronger than in retinal cells [90] . 

Virtually all parvocellular cells (99%) present linear spatial summation. That is, 
the response to two elements presented simultaneously to the receptive field equals 
the sum of the response to each of the elements presented separately. About 75% 
of magnocellular cells are also linear, the other 25% are not [101]. 

The LGN is often called a relay nucleus because it is the only structure 
between the retina and the cortex. However, LGN neurons are part of a complex 
circuit that involves ascending, descending and recurrent sets of neuronal con- 
nections [5, 194, 201]. The major source of descending input comes from neu- 
rons in layer 6 of VI. These feedback connections can be excitatory (through 
direct monosynaptic connections) or inhibitory (through inhibitory interneurons 
in the LGN or the reticular nucleus of the thalamus) [67, 83]. The functions of 
the corticothalamic pathway are still under discussion [5]. These connections 
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could help to explain LGN neurons' extra-classical receptive field properties, 
such as the effects of suppressive field [5, 27, 38, 158]. It is generally agreed that 
these feedback connections act by modulating the responsiveness of the LGN 
neurons, and not by driving the actual responses [193]. It is possible that the 
major role of feedback in the visual system is to maintain top-down attention 
[124, 125]. 



2.4 VI 

2.4.1 Anatomy 

LGN neurons send their axons through the optic radiations to the back of the brain, 
where the primary visual cortex, area VI, is located. VI is virtually the only target 
of primate LGN neurons [19, 35]. The magnocellular and parvocellular pathways 
that started in the retina remain largely separated. 

VI, like most cortical areas, has six main layers [31]. Most of the LGN inputs 
arrive to layer 4, which is divided in four sublayers: sublayer 4Ca receives axons 
mostly from magnocellular neurons. Sublayer 4CP (and sublayer 4A to a lesser 
extent) receives axons mostly from parvocellular neurons. Layer 6 receives weak 
input from collaterals of the same LGN axons that provide strong input to layer 4C 
[22, 85, 94, 116]. Neurons from the koniocellular layers in the LGN send their 
axons to layer 1 and layers 2-3 [87, 112]. 

Layer 4Ca sends its output to 4B [37, 72, 1 17]. Axons from neurons in 4CP 
terminate in the deepest part of layer 3 [37, 72, 107]. Layers 2, 3, and 4B proj- 
ect mainly to other cortical regions [36] and also send axons to layer 5 [23]. 
Layer 5 projects back to layers 4B, 2, 3 [37] and to the superior colliculus 
[118]. Layer 6 projects to the LGN [73, 227] and also sends axons to several 
VI layers [117, 227]. Many of the projection pyramidal cells in layers 2, 3, 4B, 
5 and 6 have collaterals that connect locally. Layer 1 contains few cell bodies, 
but many axons and dendrites synapse there [119]. Figure 2.4 shows a sche- 
matic representation of the main connections. 

In addition to the feedforward input coming from the LGN, VI receives direct 
feedback from areas V2, V3, V4, V5 (or MT), MST, FEF, LIP and inferotemporal 
cortex [17, 169, 184, 196, 203, 213, 214]. The projections from these areas termi- 
nate in layers 1, 2, and 5 of VI, with occasional arbors in layer 3 [185, 197]. 



2.4.2 Physiology and Receptive Fields 

In primates, the receptive fields of most VI input neurons (layer 4C) have the same 
center-surround organization as the LGN neurons they receive direct input from 
[18, 21, 34, 113, 172]. Outside of layer 4C, the receptive field structure is very 
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Fig. 2.4 Schematic representation of VI inputs, outputs and vertical interconnections, (a) From 
[88]. (b) From [36] 
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Fig. 2.5 (a) Schematic representation of simple cell receptive fields with different orientations 
and number of subregions. (b) Receptive field selective to vertical orientations. A vertical light bar 
over the excitatory region is the optimal stimulus (left). A non-vertical light bar (right) that par- 
tially falls on the inhibitory regions makes the cell fire less, (c) Cell stimulated with a bar of the 
preferred spatial frequency (left) and with a bar that is too wide and thus falls on the opposite 
contrast subregions 



different and we can distinguish two main groups of cells according to their receptive 
field type: simple cells and complex cells [91]. 

Simple cells: Hubel and Wiesel first described the receptive fields of "simple cells" 
in area VI [89]. The receptive fields of simple cells are organized in distinct 
elongated on and off antagonistic subregions, whose spatial arrangement deter- 
mines the responses of the neuron to different stimuli. Simple cells are selective to 
the orientation and spatial frequency of the stimulus (Fig. 2.5). The response of 
simple neurons is reduced when there is a mismatch between the light and dark 
parts of the stimulus and the on- and off -regions of the receptive field. By testing 
the neuron's responses to different stimuli, it is possible to generate tuning curves 
for orientation and spatial frequency. 

Hubel and Wiesel [9 1 ] proposed that each simple cell gets its input from an array 
of center-surround receptive fields of the same sign that have their centers arranged 
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Fig. 2.6 Schematic representation of the feed forward excitatory model proposed by Hubel and 
Wiesel in 1962. From [91] 



Fig. 2.7 A complex cell 
gives the same response to 
bars anywhere within the 
receptive field, and does not 
prefer either light or dark bars 
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along a straight line on the retina. The synapses from the center-surround receptive 
fields to the simple cell are excitatory and this gives the simple receptive fields its 
elongated shape and orientation selectivity (Fig. 2.6). Recent studies have provided 
strong support for this model [9, 69-71, 180, 216]. 

Complex cells: complex cells in the primary visual cortex, discovered by Hubel 
and Wiesel, are selective to the orientation and spatial frequency of stimuli (like 
simple cells) but their receptive fields do not have distinctive on and off subregions 
[91]. Consequently, complex receptive fields are invariant to the spatial phase (posi- 
tion of the stimulus within the receptive field) and contrast polarity of the stimulus. 
When a single bar is presented within the receptive field, complex cells respond 
equally well regardless of the bar's position and contrast, as long as the bar has the 
preferred orientation and width (Fig. 2.7). When pairs of bars are presented simul- 
taneously within the receptive field, complex cells exhibit nonlinearity in spatial 
summation [91]: the response to simultaneous presentation of two stimuli cannot 
be predicted from the sum of the responses to the two stimuli presented individually. 
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Fig. 2.8 Different hypothesis about the connectivity of complex cells. After [142] 



This is a fundamental property of complex cells; simple cells are more or less linear 
[39,91, 155, 182]. 

The circuits that gives rise to complex cells is not fully understood; there are 
several different hypotheses in the literature, some of which are shown in Fig. 2.8. 
The "cascade model" [91] suggests that simple cells and complex cells represent 
two successive stages in hierarchical processing: in a first stage, simple cells are 
created from the convergence of center-surround inputs that have receptive fields 
aligned in visual space. In the second stage, complex cells are then generated by the 
convergence of simple cells inputs with similar orientation preferences (Fig. 2.8, 
left). "Parallel models" [202] propose that simple cells and complex cells are both 
constructed from direct geniculate inputs. Simple cells are created from the conver- 
gence of linear LGN inputs, and complex cells from the convergence of non-linear 
LGN inputs (Fig. 2.8, middle). "Recurrent models" [40] use a combination of weak 
simple cell inputs and strong recurrent complex cell inputs to generate complex cell 
nonlinearities (Fig. 2.8, right). Martinez and Alonso [8, 142, 143] published 
evidence supporting the Hubel and Wiesel cascade model. 

End-stopped cells: ordinary simple and complex cells show length summation: the 
longer the bar stimulus, the better the response, until the bar is as long as the 
receptive field; making the bar even longer has no further effect. For end-stopped 
cells, lengthening the bar improves the response up to some limit, but exceeding 
that limit in one or both directions results in a weaker response. The same stimulus 
orientation evokes maximal excitation on the activating region and maximal inhibi- 
tion on the outlying areas. Hubel and Wiesel discovered and characterized end- 
stopped cells in cat areas 18 and 19 and initially called them hypercomplex cells 
[92]. Later Gilbert showed that some simple and complex cells in cat area 17 are 
also end-stopped [25, 78]. Several recent studies suggest that most primate VI cells 
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Fig. 2.9 A curved border 
would be a good stimulus for 
the end-stopped cell repre- 
sented in the diagram. From 
[88] 
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are somewhat end-stopped [97, 100, 103, 165, 188]. The receptive field structure of 
end-stopped cells makes them especially sensitive to corners, curvature and termi- 
nators [88, 92] (Fig. 2.9). 

Columnar organization: A fundamental feature of cortical organization is the 
spatial grouping of neurons with similar properties. VI is functionally organized in 
layers and cortical columns, which are roughly perpendicular to the layers. The 
concept of cortical columns was introduced by Mountcastle in the somatosensory 
system [153, 154, 174], although Lorente de No had envisaged their existence 
through his anatomical studies [1 14]. Hubel and Wiesel discovered columnar orga- 
nization in area VI, first in the cat [91] and then in the primate [93, 95, 226]. They 
showed that VI cells with similar properties are grouped into columns: as they 
advanced an electrode in an orthogonal penetration from the cortex surface, 
they found that the neurons recorded by the electrode had similar receptive field 
axis orientation, ocular dominance, and position in the visual field. 

- Ocular dominance columns: the inputs from the two eyes are segregated in layer 4, 
where cortical neurons are driven monocularly. In any given column extending 
above and below layer 4, all the cortical neurons, even if driven by both eyes, 
share the same eye preference. Ocular dominance columns form an interdigitating 
pattern on the cortex [91, 93, 226]. Figure 2.10 shows an ocular dominance map 
obtained with intrinsic optical imaging: we can see distinct strips in a 1 cm 2 
patch of cortex, activated by a stationary bar presented monocularly to the visual 
system of a rhesus monkey. 

- Orientation columns: Hubel and Wiesel [91, 93, 95] found that, just as with eye 
dominance, orientation preference remains constant in orthogonal penetrations 
through the cortical surface: the cortex is subdivided into narrow regions of 
constant orientation, extending from the surface to the white matter but inter- 
rupted by layer 4C, where most cells have no orientation preference [18, 21, 34, 
113, 172] (although some recent studies have found orientation selective cells in 
layer 4C [84, 181, 191]). In a tangential electrode penetration, the orientation 
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Fig. 2.10 Al cm 2 image from 
cortical area V 1 in a primate. 
The stripes indicate an ocular 
dominance map created when 
visual stimuli are displayed 
to the right eye versus the left 
eye [121] 





Fig. 2.11 An orientation map of the V1/V2 border from a cat (VI and V2 are called area 17 and 
18 in cats, by convention) obtained with intrinsic optical imaging. The legend on the right shows 
the relationship between the color of each pixel and orientation. The brightness of each pixel 
indicates the selectivity of each point in the map: dark indicates points in the map that are not 
particularly selective to any orientation, while bright points signify points in the map that are 
tuned specifically to a given orientation [ 1 27] 



preference usually changes gradually. Figure 2.11 shows an orientation selectivity 
map in areas 17 and 18 of the cat visual cortex (equivalent to areas VI and V2 
in the primate) obtained with optical intrinsic signal: the image shows the prefer- 
ence of neurons to lines of different orientations, when presented to the retina. 
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Optical imaging studies have provided precise details about the columnar 
organization: orientation columns are arranged radially into pinwheel-like struc- 
tures with orientation preference shifting gradually along contours circling the 
pinwheel center [20, 26]. Each pinwheel center tends to occur near the center of an 
ocular dominance patch [43, 1 15], and iso-orientation contours tend to cross ocular 
dominance boundaries at right angles [162]. Cortical columns where orientation 
preference changes smoothly or remains essentially constant are interspersed with 
regions containing orientation singularities where the orientation changes abruptly 
by up to 90° [24, 26, 54, 95, 211]. 

Horizontal and feedback connections: Many local connections in VI have a wide 
lateral distribution, including long intralaminar connections spreading several 
millimeters [36]. Prominent horizontal connections are those originating from and 
terminating in layers 2-3 and 4B; these connections arise from neurons whose 
long-distance axon collaterals form periodic clusters [10, 37, 80, 81, 146, 183]. 
These clusters tend to preferentially link columns of neurons with similar response 
properties: in cats, ferrets, and monkeys they preferentially link columns with similar 
orientation preference [130, 212]. Feedback connections from extrastriate cortex to 
VI also show an orderly topographic organization and terminate in a patch-like 
manner within VI [11]. These two types of orderly connections (horizontal and 
feedback) may be involved in the generation of suppressive fields in VI neurons, 
as well as other extra-classical receptive field modulations [11, 38, 39, 79, 110]. 
Intra cortical connections may be important to understand the neural computations 
carried out in VI. Zhaoping has proposed that VI creates a saliency map using intra 
cortical mechanisms. This saliency map can be used to attract attention to a visual 
location without top-down factors, which may explain certain visual search proper- 
ties [233]. Macknik and Martinez-Conde have proposed that the primary role of 
feedback may be the maintenance of top-down attention [124, 125]. 



2.5 Extrastriate Cortex: The Dorsal and Ventral Visual 
Pathways 

The primate cortex has at least 32 distinct visual areas [64, 68] (Fig. 2.12). 

In the first two stages of cortical processing (VI and V2), the magnocellular and 
the parvocellular pathways are largely segregated: inputs from the LGN arrive to 
different sublayers in VI according to their magno/parvo origin and projections 
from V 1 layer 4C are also fairly separated in V 1 and V2 as revealed by cytochrome 
oxidase staining [111, 113, 163, 219]. After VI there are two main processing 
streams, associated with different visual capabilities [64, 144, 215]: 

- The dorsal or parietal stream is tuned to moving stimuli (with similar properties 
to the magnocellular pathway). After V2 the information flows to MT, MST and 
other intermediate areas. MT neurons are selective to the direction of stimulus 
motion, speed and binocular disparity [4, 13, 229, 230]. The highest stages of 
this stream are clustered in the posterior parietal cortex. This stream is involved 
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Fig. 2.12 Visual areas in the primate shown in a flattered brain. From [218] 



in assessment of spatial relationships and it is often called the "Where" 
pathway. 
- The ventral or temporal stream emphasizes form and color analysis (similar 
properties as the parvocellular pathway). After V2 the information flows to V4 
and other intermediate areas; many V4 neurons are selective to stimulus color 
[231, 232], orientation, width, and length of bars [61], curvilinear and linear 
gratings [74, 75], and contour features like angles and curves [166]. The highest 
stages of this stream are clustered in the inferotemporal cortex. This stream is 
concerned with visual recognition of objects as it is often called the "What" 
pathway. 

The transformations of the visual image that occur along each of these pathways 
do not appear to result in increased selectivity for basic parameters [145] such as 
direction or speed [4] in the dorsal pathway or wavelength [56] or orientation [62] 
in the ventral pathway. Rather than sharpening basic tuning curves, the transformation 
of information along each of the pathways appears to construct new, more complex 
response properties; both pathways may use similar computational strategies for pro- 
cessing information [145]. Also, retinotopic specificity decreases progressively in suc- 
cessive levels of each of the pathways: the average receptive field size in MT is 100 times 
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Fig. 2.13 Schematic of the two visual pathways in the primate, showing the main connections 
between the different areas. From [98] 

larger than in VI [76]. In MST receptive fields can cover a full quadrant of the visual 
field [63]. V4 receptive fields are about 30 times larger than VI receptive fields [129, 
220], and downstream in the ventral pathway they become over 100 times larger [60]. 

The hypothesis of two distinct streams of processing was initially formulated by 
Ungerleider and Mishkin [215]. Many different groups have provided anatomical, 
physiological, and behavioral support to this idea. In humans, clinical observations 
indicate that damage to the parietal cortex can affect visual perception of position, 
leaving object recognition unimpaired [52, 178, 234]. Temporal lobe lesions can 
produce specific deficits related to object recognition [53, 147, 148, 167]. 
Systematic lesion studies in primates have found a functional separation between 
the temporal and the parietal cortices [58, 151, 213]. 

While it is widely accepted that information is computed in these two largely 
parallel visual pathways (as shown in schematic on Fig. 2.13 taken from [98]), it is 
important to note that the separation between the two pathways is far from com- 
plete. There is anatomical and physiological evidence of substantial cross-talk 
between the two streams [68, 149, 218]. 



2.6 The Role of Spatiotemporal Edges in Early Vision 



Information flows from one visual area to the next in the form of excitatory signals 
carried through glutamate synapses. Therefore, all inhibition between neurons, 
for instance to form receptive fields, is a function of local inhibitory circuits. 
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Local inhibition, which underlies center-surround receptive field organization, is 
enacted through the neurotransmitter GABA (gamma-aminobutyric acid). In 1965 
Hartline and Ratliff delineated the far-reaching consequences of this simple 
arrangement, in terms of spatial and temporal visual processing [179]. They showed 
that the three components of a laterally inhibitory circuit: 

1 . Excitatory input and output: information arrives at a given visual area of the brain 
in the form of excitatory neural responses, and the information is sent to the next 
visual area(s) in the visual hierarchy as excitatory neural responses as well. 

2. Lateral inhibition: occurs as a function of excitatory activation (thus inhibition 
follows excitation in time). 

3. Self-inhibition: neurons that laterally inhibit their neighbors also inhibit themselves. 

Figure 2. 14 shows a plausible mammalian descriptive model of lateral inhibition, 
based on Hartline and Ratliff's original Limulus model [123]. The model predicts 
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Fig. 2.14 A mammalian representation of the spatial lateral inhibition model originally proposed 
by Hartline and Ratliff. The excitatory neurons in the center of the upper row receive excitatory 
input from a visual stimulus. This excitation is transmitted laterally to the inhibitory neurons just 
outside the stimulus, and also within the area impinged upon by the stimulus. The inhibitory 
interactions between excited neurons at the edges of stimuli and their non-excited neighbors 
results in apparent contrast enhancement at the borders of the stimulus. Output of each of the 
excitatory neurons is represented in action potentials per unit time at the bottom [127] 
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Fig. 2.15 This Mach Band demonstration was originally designed by Chevreul in 1839. Notice 
how each vertical stripe appears to be lighter on the left than on the right. This illusory effect is 
due to contrast enhancement at the borders 



that the strongest neural excitatory signals to a visual stimulus will occur just inside 
the stimulus' spatial borders. Neural inhibition, moreover, is strongest just outside 
of the borders. The spatial interiors of stimuli do not cause responses in visual 
neurons. It is hypothesized that the interiors of large spatial stimuli are visible 
through the illusory process of filling-in. One perceptual consequence of lateral 
inhibition is that stimuli to both sides of a luminance border are differentially 
enhanced in an illusory fashion (as in Fig. 2.15). 

If we now examine two of the neurons in a lateral inhibitory network through 
time, one neuron being excitatory and the other inhibitory, we should expect the 
following specific temporal pattern of response (Fig. 2.16). Visual information 
enters a given visual area as excitatory input to specific neurons that are tuned to 
the specific visual stimulus being presented. The excited neurons then locally 
inhibit their neighbors, and also themselves, in a delayed inhibitory response that 
serves to bring suppress the initial transient onset-response. This state of excitatory- 
inhibitory equilibrium continues until such point that the excitatory input representing 
the stimulus is extinguished. After that point, the neurons briefly enter a state of 
suppression due to the fact that delayed inhibition is unopposed by excitation 
(a refractory period called the time-out), followed by a disinhibitory rebound, 
called an after-discharge. Just as neurons respond strongly to the spatial borders of 
stimuli due to lateral inhibition, so too do they respond strongly to the temporal 
borders (the stimulus onsets and terminations (also commonly called "stimulus 
offsets," although this term is linguistically incorrect)). The perceptual result of this 
is contrast enhancement at the temporal borders of stimuli. Lateral inhibition is thus 
responsible not only for the spatial layout of receptive fields, but also for their temporal 
response properties. The perceptual result of this process is that the perceived 
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Fig. 2.16 One excitatory and one inhibitory neuron, followed through a period of time in which the 
stimulus is off (times 1, 2 and 3), on (times 4, 5, 6, and 7), and then off (times 8, 9 and 10) [127] 



contrast of a stimulus is highest just after it turns on and then again after it turns 
off. Visual masking (the effect in which the visibility of a target stimulus is reduced 
by a masking stimulus that does not overlap the target in space or time) occurs 
perceptually when the neural responses to the target onset and/or termination are 
inhibited, suggesting that the onset-response and after-discharge are critical for the 
visibility of stimuli [120, 122, 126]. 



2.7 The Role of Corners in Early Vision 



2.7.1 Overview 



Our perception of the visual world is constructed, step-by-step, by neurons in dif- 
ferent visual areas of the brain [59, 68, 91, 195]. While feedback certainly plays a 
role in the visual system [6, 7, 96, 124, 125, 133, 157], the visual system's overall 
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tendency is towards a hierarchy, in which neurons in sequential levels extract more 
and more complicated features from the visual scene. These features include (but 
are not limited to) color, brightness, movement, shape, and depth. 

In order to determine how visual perception is constructed in our brain, we need 
first to establish the nature of the fundamental visual features in a scene. Theories of 
shape and brightness perception have primarily focused on the detection and 
processing of visual edges. Early visual neurons are thought of as "edge detectors" 
[91, 132], and current studies are based on the assumption that edges are the most 
elementary visual feature. However, recent experiments show that corners can be 
more salient than edges, both perceptually (Fig. 2. 17) [206, 208] and in the responses 




Fig. 2.17 Vasarely's nested squares and alternating brightness star illusions, (a) Nested squares 
illusion, based in Vasarely's "Arcturus" [221]. Top: The stimulus is made out of multiple concen- 
tric squares of increasing luminance (going from black in the center to white in the outside). The 
two circles indicate two regions that appear to have significantly different brightness. The area 
inside the upper circle has higher average luminance than the region inside the lower circle; how- 
ever the region inside the upper circle appears perceptually darker. Bottom: Nested square stimu- 
lus, with a gradient of decreasing luminance (from the center to the outside). From [206]. (b, c) 
The Alternating Brightness Star illusion [134]. The stimulus is made of concentric stars of graded 
luminance. In the examples illustrated, the innermost star is white; the outermost star is black. The 
illusory corner-folds that radiate from the center appear as light or dark depending on the polarity 
of the corner angle; Corner Angle Brightness Reversal effect. Moreover, the illusory folds appear 
more salient with sharp corners (top stars), and less salient with shallow corners {bottom stars); 
Corner Angle Brightness Variation effect. However, all illusory folds are physically equal to each 
other in luminance, (b) The gradient from the center to the outside has ten luminance steps, and 
so the individual stars forming the polygonal constructs are easy to identify, (c) The gradient from 
the center to the outside has 100 luminance steps. From [208] 
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Fig. 2.18 Center-surround receptive field responses to corners of varying angles, (a) 
Computational simulations with a DOG filter. The filter parameters were chosen to match 
physiological center-surround receptive fields at the eccentricity used in the psychophysical 
experiments (3°). Top: Examples of corner-gradient stimuli analyzed in the simulations. The 
circles mark the point of 50% luminance. Bottom: Convolving the DOG filter with the stimuli 
in (A) simulates the output of an array of center-surround neurons. The circles indicate the 
responses of the model at the point of 50% luminance on the actual gradient, (b) Generalized 
model of corner processing. Three on-center receptive fields are respectively placed over one 
edge and two corners of a white triangle. The center of the receptive field over the edge (posi- 
tion A) is well stimulated by light, but most of the surround also falls in the light region, so the 
response of the neuron is partially inhibited. The center of the receptive field over the 90° corner 
(position B) is also stimulated by light and most of the surround falls in the dark area. This is a 
more optimal stimulus than in (A) and leads to a stronger neural response. The receptive field 
over the 45° corner (position C) receives even more optimal contrast between center and sur- 
round, leading to an even stronger response. The spiking responses depicted in the cartoon are 
hypothetical. From [206] 



of neurons throughout the visual hierarchy, even in early stages (Fig. 2.18) [206, 
210]. Combined results from human psychophysics experiments, human brain 
imaging, and computational modeling suggest that deflections or discontinuities 
in edges, such as corners, curvature, and terminating line endings, may be first 
processed by center-surround receptive fields [206, 208, 210]. These data 
suggest that corners may be a fundamental feature for shape and brightness 
perception. 

This hypothesis in no way rules out a critical role for later cortical areas in more 
complex processing of corner angles. For instance, specific orientations of corner 
angles must be processed cortically, given that the first orientation-selective cells 
are cortical. 
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2. 7.2 Corner Perception and the Redundancy-Reducing 
Hypothesis 

The information transmitted by our visual system is constrained by physical 
limitations, such as the relatively small number of axons available in the optic 
nerve. To some extent, our visual system overcomes these limitations by extracting, 
emphasizing, and processing non-redundant visual features. In 1961, Barlow 
proposed that the brain recodes visual data "so that their redundancy is reduced 
but comparatively little information is lost." This idea is known as the 
"Redundancy-Reducing Hypothesis" [14, 15]. The redundancy-reducing hypoth- 
esis has been invoked as an explanation for why neurons at the early levels of 
the visual system are suited to perform "edge-detection," or "contour-extraction." 
However, redundancy reduction is not necessarily constrained to edges, but 
rather should theoretically apply to any feature in the visual scene [177]. Just as 
edges are a less redundant feature than diffuse light, Fred Attneave proposed in 
the 1950s that "points of maximum curvature" (i.e., discontinuities in edges, 
such as curves, angles and corners - any point at which straight-lines are deflected) 
are even less redundant than edges themselves, and thus contain more informa- 
tion [12], If points of high curvature are less redundant than points of low 
curvature, then sharp corners should also be less redundant than shallow corners. 
This hypothesis is consistent with experiments showing that sharp corners are 
perceptually more salient and generate stronger physiological responses than 
shallow corners [206, 208, 210]. 



2.8 Effects of Fixational Eye Movements in Early Visual 
Physiology and Perception 

2.8.1 Overview 

As we read a page of text, our eyes rapidly flick from left to right in small hops, 
bringing each word sequentially into focus. When we look at a person's face, our 
eyes similarly dart here and there, resting momentarily on one eye, the other eye, 
mouth and other features. But these large eye movements, called saccades 
(Fig. 2.19a), are just a small part of the daily workout our eye muscles get. Our eyes 
never stop moving: even when they are apparently fixated on something, they still 
jump and jiggle imperceptibly in ways that turn out to be essential for seeing. The 
tiny eye motions that we produce whenever we fixate our gaze are called fixational 
eye movements (Fig. 2.19b) [139]. If these miniature motions are halted during 
fixation, all stationary objects simply fade from view. 
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Fig. 2.19 Fixational eye movements and visual fading, (a) An observer views a picture (left) 
while eye positions are monitored (right). The eyes jump, seem to fixate or rest momentarily, 
producing a small dot on the trace, then jump to a new region of interest. The large jumps in eye 
position illustrated here are called saccades. However, even during fixation, or "rest" times, eyes 
are never still, but continuously produce fixational eye movements: drifts, tremor, and microsac- 
cades. From [228]. (b) Cartoon representation of fixational eye movements in humans and pri- 
mates. Microsaccades (straight and fast movements), drifts (curvy slow movements) and tremor 
(oscillations superimposed on drifts) transport the visual image across the retinal photoreceptor 
mosaic. From [135]. (c) Troxler fading. In 1804 Swiss philosopher Ignaz Paul Vital Troxler dis- 
covered that deliberately fixating on something causes surrounding stationary images to fade 
away. To elicit this experience, stare at the central dot while paying attention to the surrounding 
pale ring. The ring soon vanishes, and the central dot appears set against a while background. 
Move your eyes, and it pops back into view. Modified from [139]. (d) This drawing illustrates the 
suction cup technique, used by Yarbus [228] and others. This technique was very popular in early 
retinal stabilization studies for its simplicity, but it is now considered old-fashioned, and other, less 
invasive stabilization techniques are preferred. The target image is directly attached to the eyeball 
by means of a contact lens assembly. The target is viewed through a powerful lens. The assembly 
is firmly attached to the eye by a suction device. Modified from [139] 



2.8.2 Neural Adaptation and Visual Fading 



That the eyes move constantly has been known for centuries. In 1860 Hermann von 
Helmholtz pointed out that keeping one's eyes motionless was a difficult proposition 
and suggested that "wandering of the gaze" prevented the retina from becoming tired. 
Animal nervous systems may have evolved to detect changes in the environ- 
ment, because spotting differences promotes survival. Motion in the visual field 
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may indicate that a predator is approaching or that prey is escaping. Such changes 
prompt visual neurons to respond with neural impulses. Unchanging objects do not 
generally pose a threat, so animal brains - and visual systems - did not evolve to 
notice them. Frogs are an extreme case, as they produce no spontaneous eye move- 
ments in the absence of head movements. For a resting frog, such lack of eye 
movements results in the visual fading of all stationary objects. Jerome Lettvin and 
colleagues stated that a frog "will starve to death surrounded by food if it is not 
moving." Thus a fly sitting still on the wall will be invisible to a resting frog, but 
once the fly is aloft, the frog will immediately detect it and capture it with its tongue. 

Frogs cannot see unmoving objects because an unchanging stimulus leads to 
"neural adaptation." That is, under constant stimulation, visual neurons adjust their 
gain as to gradually stop responding. Neural adaptation saves energy but also limits 
sensory perception. Human neurons also adapt to sameness. However, the human 
visual system does much better than a frog's at detecting unmoving objects, 
because human eyes create their own motion, even during visual fixation. Fixational 
eye movements shift the visual scene across the retina, prodding visual neurons into 
action and counteracting neural adaptation. They thus prevent stationary objects 
from fading away. 

The goal of oculomotor fixational mechanisms may not be retinal stabilization, 
but rather controlled image motion adjusted so as to overcome adaptation in an 
optimal fashion for visual processing [198]. 

In 1804, Troxler reported that precisely fixating the gaze on an object of interest 
causes stationary images in the surrounding region gradually to fade away. Thus, 
even a small reduction in the rate and size of fixational eye movements greatly 
impairs vision, even outside of the laboratory and for observers with healthy eyes 
and brains (Fig. 2.19c). 

Eliminating all eye movements, however, can only be achieved in a laboratory. In 
the early 1950s, some research teams achieved this stilling effect with a tiny custom 
slide projector, mounted directly onto a contact lens that attached directly to the 
observer's eye with a suction device (Fig. 2.19d). In this setup, a person views the 
projected image through this lens, which moves with the eye. Using such a retinal 
stabilization technique, the image shifts every time the eye shifts. Thus it remains still 
with respect to the eye, causing the visual neurons to adapt and the image to fade away. 
Nowadays, researchers create this same result by measuring eye movements with a 
camera pointed at the eye. They transmit the eye-position data to a projection system 
that moves the image with the eye, thereby stabilizing the image on the retina. 

Around the same time, three different types of fixational eye movements were 
characterized. Microsaccades are small, involuntary saccades that are produced 
when the subjects attempt to fixate their gaze on a visual target. They are the largest 
and fastest of the fixational eye movements, carrying an image across dozens to 
several hundreds of photoreceptors. Drifts are slow meandering motions that occur 
between the fast, linear microsaccades. Tremor is a tiny, very fast oscillation super- 
imposed on drifts. Tremor is the smallest type of fixational eye movement, its 
motion no bigger than the size of one photoreceptor. See Martinez-Conde et al. 
[136, 139, 141] for some recent reviews of fixational eye movement parameters in 
humans, primates, and other vertebrates. 
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2.8.3 Microsaccades in Visual Physiology and Perception 

Starting in the late 1990s, fixational eye movement research has focused on 
microsaccades. Physiological experiments found that microsaccades increase the 
firing of neurons in the visual cortex and lateral geniculate nucleus, by moving the 
images of stationary stimuli in and out of neuronal receptive fields. Firing rate 
increases following microsaccades were clustered in bursts of spikes, whereas 
individual spikes tended to occur in the periods between microsaccades. Moreover, 
bursts of spikes were better correlated with previous microsaccades than either 
single spikes or instantaneous firing rate. Bursts highly correlated with previous 
microsaccades had large spike numbers and short inter-spike intervals [137, 138]. 
Because microsaccades are related to maintaining visibility and counteracting fad- 
ing (see further below), bursts that indicate previous microsaccades accurately 
must encompass the neural code for visibility. In area VI, optimal burst sizes fol- 
lowing microsaccades tended to be three spikes or more. These bursts may be an 
important clue to the neural code or "language" that our brain uses to represent the 
visibility of the world [137]. The neural codes by which neurons, or neuronal 
populations, encode and transmit visual information are not only critical to our 
understanding of normal visual processing, but also to the development and refine- 
ment of neural prostheses. 

Microsaccades could enhance spatial summation by synchronizing the activity 
of nearby neurons [137]. By generating bursts of spikes, microsaccades may also 
enhance temporal summation of responses from neurons with neighboring RFs 
[137]. Moreover, microsaccades may help disambiguate latency and brightness in 
visual perception, allowing us to use latency in our visual discriminations [137]. 
Changes in contrast can be encoded as changes in the latency of neuronal responses 
[2, 3, 77]. Since the brain knows when a microsaccade is generated, differential 
latencies in visual responses could be used by the brain to indicate differences in 
contrast and salience. 

Despite several decades of debate (see [139] for a review), a direct link between 
microsaccade production and visual perception has only recently been demon- 
strated. Martinez-Conde et al. [140] found that increased microsaccade production 
during fixation resulted in enhanced visibility for visual targets. Conversely, 
decreased microsaccade production led to periods of visual fading. These results 
established a potential causal relationship between microsaccades and target visi- 
bility during fixation, and corroborated predictions from previous physiological 
studies in which microsaccades were found to increase the spiking rates in visual 
neurons [137, 138]. Microsaccade production has been subsequently linked to 
perceptual transitions in various other visual phenomena, such as binocular rivalry 
[215], filling-in of artificial scotomas [207], and illusory motion (perceived speed 
as well as subjective direction [108, 209]). 

Fewer studies have addressed the neural and perceptual consequences of drifts 
and tremor. However, all fixational eye movements may contribute significantly to 
visual perception, depending on stimulation conditions. For example, receptive 
fields in the periphery may be so large that only microsaccades are large and fast 
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enough - compared to drifts and tremor - to prevent visual fading, especially with 
low-contrast stimuli. Whereas foveal receptive fields may be so small that drifts and 
tremor can maintain vision in the absence of microsaccades. But even if drifts and/ 
or tremor can maintain foveal vision on their own, this does not rule out that 
microsaccades could also have a role. Thus, if one were to eliminate drifts and 
tremor, microsaccades alone might sustain foveal vision during fixation. 



References 



1. Ahnelt PK, Kolb H, Pflug R (1987), Identification of a subtype of cone photoreceptor, likely 
to be blue sensitive, in the human retina. J Comp Neurol, 255(1): p. 18-34. 

2. Albrecht DG (1995), Visual cortex neurons in monkey and cat: effect of contrast on the 
spatial and temporal phase transfer functions. Vis Neurosci, 12(6): p. 1191-210. 

3. Albrecht DG, Hamilton DB (1982), Striate cortex of monkey and cat: contrast response 
function. J Neurophysiol 48: p. 217-37. 

4. Albright TD (1984), Direction and orientation selectivity of neurons in visual area MT of the 
macaque. J Neurophysiol, 52(6): p. 1106-30. 

5. Alitto HJ, Usrey WM (2003), Corticothalamic feedback and sensory processing. Curr Opin 
Neurobiol, 13(4): p. 440-5. 

6. Alonso JM, Cudeiro J, Perez R, et al. (1993), Influence of layer V of area 18 of the cat visual 
cortex on responses of cells in layer V of area 17 to stimuli of high velocity. Exp Brain Res, 
93(2): p. 363-6. 

7. Alonso JM, Cudeiro J, Perez R, et al. (1993), Orientational influences of layer V of visual 
area 18 upon cells in layer V of area 17 in the cat cortex. Exp Brain Res, 96(2): p. 212-20. 

8. Alonso JM, Martinez LM (1998), Functional connectivity between simple cells and complex 
cells in cat striate cortex. Nat Neurosci, 1(5): p. 395-403. 

9. Alonso JM, Usrey WM, Reid RC (2001), Rules of connectivity between geniculate cells and 
simple cells in cat primary visual cortex. J Neurosci, 21(11): p. 4002-15. 

10. Anderson JC, Martin KA, Whitteridge D (1993), Form, function, and intracortical projec- 
tions of neurons in the striate cortex of the monkey Macacus nemestrinus. Cereb Cortex, 
3(5): p. 412-20. 

1 1 . Angelucci A, Levitt JB, Walton EJ, et al. (2002), Circuits for local and global signal integra- 
tion in primary visual cortex. J Neurosci, 22(19): p. 8633^-6. 

12. Attneave F (1954), Some informational aspects of visual perception. Psychol Rev, 61(3): 
p. 183-93. 

13. Baker JF, Petersen SE, Newsome WT, Allman JM (1981), Visual response properties of 
neurons in four extrastriate visual areas of the owl monkey (Aotus trivirgatus): a quantitative 
comparison of medial, dorsomedial, dorsolateral, andmiddle temporal areas. J Neurophysiol, 
45(3): p. 397^116. 

14. Barlow HB (1961), Possible principlesunderlying the transformation of sensory messages, in 
Sensory Communication, Rosenblith WA, Editor. MIT Press: Cambridge, MA. p. 217-34. 

15. Barlow HB (1989), Unsupervised learning. Neural Computation, 1: p. 295-311. 

16. Barlow HB, Fitzhugh R, Kuffler SW (1957), Change of organization in the receptive fields 
of the cat's retina during dark adaptation. J Physiol, 137: p. 228-54. 

17. Barone P, Batardiere A, Knoblauch K, Kennedy H (2000), Laminar distribution of neurons 
in extrastriate areas projecting to visual areas VI and V4 correlates with the hierarchical 
rank and indicates the operation of a distance rule. J Neurosci, 20(9): p. 3263-81. 

18. Bauer R, Dow BM, Vautin RG (1980), Laminar distribution of preferred orientations in 
foveal striate cortex of the monkey. Exp Brain Res, 41(1): p. 54-60. 



2 Vision's First Steps: Anatomy, Physiology, and Perception in the Early Visual System 49 

19. Benevento LA, Standage GP (1982), Demonstration of lack of dorsal lateral geniculate 
nucleus input to extrastriate areas MT and visual 2 in the macaque monkey. Brain Res, 
252(1): p. 161-6. 

20. Blasdel G, Obermayer K, Kiorpes L (1995), Organization of ocular dominance and orientation 
columns in the striate cortex of neonatal macaque monkeys. Vis Neurosci, 12(3): p. 589-603. 

21. Blasdel GG, Fitzpatrick D (1984), Physiological organization of layer 4 in macaque striate 
cortex. J Neurosci, 4(3): p. 880-95. 

22. Blasdel GG, Lund JS (1983), Termination of afferent axons in macaque striate cortex. 
J Neurosci, 3(7): p. 1389-413. 

23. Blasdel GG, Lund JS, Fitzpatrick D (1985), Intrinsic connections of macaque striate cortex: 
axonal projections of cells outside lamina 4C. J Neurosci, 5(12): p. 3350-69. 

24. Blasdel GG, Salama G (1986), Voltage-sensitive dyes reveal a modular organization in mon- 
key striate cortex. Nature, 321(6070): p. 579-85. 

25. Bolz J, Gilbert CD (1986), Generation of end-inhibition in the visual cortex via interlaminar 
connections. Nature, 320(6060): p. 362-5. 

26. Bonhoeffer T, Grinvald A (1991), Iso-orientation domains in cat visual cortex are arranged 
in pinwheel-like patterns. Nature, 353(6343): p. 429-31. 

27. Bonin V, Mante V, Carandini M (2005), The suppressive field of neurons in lateral geniculate 
nucleus. J Neurosci, 25(47): p. 10844-56. 

28. Bowling DB (1980), Light responses of ganglion cells in the retina of the turtle. J Physiol, 
299: p. 173-96. 

29. Boycott BB, Dowling JE (1969), Organization of the primate retina: light microscopy. Philos 
Trans R Soc Lond B Biol Sci, B, 255: p. 109-84. 

30. Boycott BB, Wassle H (1991), Morphological Classification of Bipolar Cells of the Primate 
Retina. Eur J Neurosci, 3(11): p. 1069-88. 

31. Brodmann K (1909), Vergleichende Lokalisationlehre der Grosshirnrinde in ihren Prinzipien- 
Dargestellt auf Grund des Zellenbaues. Leipzig: Barth. 

32. Brown PK, Wald G (1963), Visual pigments in human and monkey retinas. Nature, 200: 
p. 37-43. 

33. Brown PK, Wald G (1964), Visual Pigments In Single Rods And Cones Of The Human 
Retina. Direct Measurements Reveal Mechanisms Of Human Night And Color Vision. 
Science, 144: p. 45-52. 

34. Bullier J, Henry GH (1980), Ordinal position and afferent input of neurons in monkey striate 
cortex. J Comp Neurol, 193(4): p. 913-35. 

35. Bullier J, Kennedy H (1983), Projection of the lateral geniculate nucleus onto cortical area 
V2 in the macaque monkey. Exp Brain Res, 53(1): p. 168-72. 

36. Callaway EM (1998), Local circuits in primary visual cortex of the macaque monkey. 
AnnuRev Neurosci, 21: p. 47-74. 

37. Callaway EM, Wiser AK (1996), Contributions of individual layer 2-5 spiny neurons to 
local circuits in macaque primary visual cortex. Vis Neurosci, 13(5): p. 907-22. 

38. Carandini M (2004), Receptive fields and suppressive fields in the early visual system, in The 
cognitive neurosciences, Gazzaniga MS, Editor. MIT Press: Cambridge, MA. 

39. Carandini M, Heeger DJ, Movshon JA (1997), Linearity and normalization in simple cells 
of the macaque primary visual cortex. J Neurosci, 17(21): p. 8621—44. 

40. Chance FS, Nelson SB, Abbot LF (1999), Complex cells as cortically amplified simple cells. 
Nature Neurosciece, 2: p. 277-82. 

41. Cicerone CM, Nerger JL (1989), The relative numbers of long-wavelength-sensitive to middle- 
wavelength-sensitive cones in the human fovea centralis. Vision Res, 29(1): p. 115-28. 

42. Conley M, Fitzpatrick D (1989), Morphology of retinogeniculate axons in the macaque. 
Vis Neurosci, 2(3): p. 287-96. 

43. Crair MC, Ruthazer ES, Gillespie DC, Stryker MP (1997), Ocular dominance peaks at 
pinwheel center singularities of the orientation map in cat visual cortex. J Neurophysiol, 
77(6): p. 3381-5. 



50 X.G. Troncoso et al. 

44. Curcio CA, Allen KA, Sloan KR, et al. (1991), Distribution and morphology of human cone 
photoreceptors stained with anti-blue opsin. J Comp Neurol, 312(4): p. 610-24. 

45. Curcio CA, Sloan KR, Jr., Packer O, et al. (1987), Distribution of cones in human and monkey 
retina: individual variability and radial asymmetry. Science, 236(4801): p. 579-82. 

46. Curcio CA, Sloan KR, Kalina RE, Hendrickson AE ( 1990), Human photoreceptor topography. 
J Comp Neurol, 292(4): p. 497-523. 

47. Dacey D, Packer OS, Diller L, et al. (2000), Center surround receptive field structure of cone 
bipolar cells in primate retina. Vision Res, 40(14): p. 1801-11. 

48. Dacey DM (1993), The mosaic of midget ganglion cells in the human retina. J Neurosci, 
13(12): p. 5334-55. 

49. Dacey DM (1999), Primate retina: cell types, circuits and color opponency. Prog Retin Eye 
Res, 18(6): p. 737-63. 

50. Dacey DM (2000), Parallel pathways for spectral coding in primate retina. AnnuRev 
Neurosci, 23: p. 743-75. 

51. Dacey DM, Petersen MR (1992), Dendritic field size and morphology of midget and parasol 
ganglion cells of the human retina. Proc Natl Acad Sci USA, 89(20): p. 9666-70. 

52. Damasio AR, Benton AL (1979), Impairment of hand movement sunder visual guidance. 
Neurology, 29(2): p. 170-4. 

53. Damasio AR, Damasio H, Van Hoesen GW (1982), Prosopagnosia: anatomic basis and 
behavioral mechanisms. Neurology, 32(4): p. 331-41. 

54. Das A, Gilbert CD (1999), Topography of contextual modulations mediated by short-range 
interactions in primary visual cortex. Nature, 399(6737): p. 655-61. 

55. De Monasterio FM, Gouras P (1975), Functional properties of ganglion cells of the rhesus 
monkey retina. J Physiol, 251(1): p. 167-95. 

56. de Monasterio FM, Schein SJ (1982), Spectral bandwidths of color-opponent cells of genicu- 
locortical pathway of macaque monkeys. J Neurophysiol, 47(2): p. 214-24. 

57. De Valois RL (1960), Color vision mechanisms in the monkey. J Gen Physiol, 43(6): p. 1 15-28. 

58. Dean P (1976), Effects of inferotemporal lesions on the behavior of monkeys. Psychol Bull, 
83(1): p. 41-71. 

59. Desimone R, Fleming J, Gross CG (1980), Prestriate afferents to inferior temporal cortex: 
an HRP study. Brain Res, 184(1): p. 41-55. 

60. Desimone R, Gross CG (1979), Visual areas in the temporal cortex of the macaque. Brain 
Res, 178(2-3): p. 363-80. 

61. Desimone R, Schein SJ (1987), Visual properties of neurons in area V4 of the macaque: 
sensitivity to stimulus form. J Neurophysiol, 57(3): p. 835-68. 

62. Desimone R, Schein SJ, Moran J, Ungerleider LG (1985), Contour, color and shape analysis 
beyond the striate cortex. Vision Res, 25(3): p. 441-52. 

63. Desimone R, Ungerleider LG (1986), Multiple visual areas in the caudal superior temporal 
sulcus of the macaque. J Comp Neurol, 248(2): p. 164-89. 

64. Desimone R, Ungerleider LG (1989), Neural mechanisims of visual processing in monkeys, in 
Handbook of neuropsychology, Boiler F, Graman J, Editors. Elsevier: Amsterdam, p. 267-99. 

65. Donner KO, Reuter T (1965), The dark-adaptation of singleunits in the frog's retina and its 
relation to the regeneration of rhodopsin. Vision Res, 5(11): p. 615-32. 

66. Dowling JE, Boycott BB (1966), Organization of the primate retina: electron microscopy. 
Proc R Soc Lond B Biol Sci, 166(2): p. 80-1 11. 

67. Erisir A, Van Horn SC, Sherman SM (1997), Relative numbers of cortical and brainstem 
inputs to the lateral geniculate nucleus. Proc Natl Acad Sci USA, 94(4): p. 1517-20. 

68. Felleman DJ, Van Essen DC (\99l),Distributed hierarchical processing in the primate cerebral 
cortex. Cereb Cortex, 1(1): p. 1 — 47. 

69. Ferster D, Chung S, Wheat H (1996), Orientation selectivity of thalamic input to simple cells 
of cat visual cortex. Nature, 380(6571): p. 249-52. 

70. Ferster D, Koch C (1987), Neuronal connectionsunderlying orientation selectivity in cat 
visual cortex. Trendes Neurosci, 10: p. 487-92. 

71. Ferster D, Miller KD (2000), Neural mechanisms of orientation selectivity in the visual 
cortex. Annu Rev Neurosci, 23: p. 441-71. 



2 Vision's First Steps: Anatomy, Physiology, and Perception in the Early Visual System 51 

72. Fitzpatrick D, Lund JS, Blasdel GG (1985), Intrinsic connections of macaque striate cortex: 
afferent and efferent connections of lamina 4C. J Neurosci, 5(12): p. 3329-49. 

73. Fitzpatrick D, Usrey WM, Schofield BR, Einstein G (1994), The sublaminar organiza- 
tion of corticogeniculate neurons in layer 6 of macaque striate cortex. Vis Neurosci, 
11(2): p. 307-15. 

74. Gallant JL, Braun J, Van Essen DC (1993), Selectivity for polar, hyperbolic, and Cartesian 
gratings in macaque visual cortex. Science, 259(5091): p. 100-3. 

75. Gallant JL, Connor CE, Rakshit S, et al. (1996), Neural responses to polar, hyperbolic, and 
Cartesian gratings in area V4 of the macaque monkey. J Neurophysiol, 76(4): p. 2718-39. 

76. Gattass R, Gross CG (1981), Visual topography of striate projection zone (MT) in posterior 
superior temporal sulcus of the macaque. J Neurophysiol, 46(3): p. 621-38. 

77. Gawne TJ, Kjaer TW, Richmond BJ (1996), Latency: another potential code for feature 
binding in striate cortex. J Neurophysiol, 76(2): p. 1356-60. 

78. Gilbert CD (1977), Laminar differences in receptive field properties of cells in cat primary 
visual cortex. J Physiol, 268(2): p. 391-421. 

79. Gilbert CD, Das A, Ito M, et al. (1996), Spatial integration and cortical dynamics. Proc Natl 
Acad Sci USA, 93(2): p. 615-22. 

80. Gilbert CD, Wiesel TN (1979), Morphology and intracortical projections of functionally 
characterised neurones in the cat visual cortex. Nature, 280(5718): p. 120-5. 

81. Gilbert CD, Wiesel TN (1983), Clustered intrinsic connections in cat visual cortex. 
J Neurosci, 3(5): p. 1116-33. 

82. Gouras P (1968), Identification of cone mechanisms in monkey ganglion cells. J Physiol, 
199(3): p. 533-47. 

83. Guillery RW, Sherman SM (2002), Thalamic relay functions and their role in corticocortical 
communication: generalizations from the visual system. Neuron, 33(2): p. 163-75. 

84. Gur M, Kagan I, Snodderly DM (2005), Orientation and direction selectivity of neurons in VI of alert 
monkeys: fiinctional relationships and laminar distributions. Cereb Cortex, 15(8): p. 1207-21. 

85. Hendrickson AE, Wilson JR, Ogren MP (1978), The neuroanatomical organization of path- 
ways between the dorsal lateral geniculate nucleus and visual cortex in Old World and New 
World primates. J Comp Neurol, 182(1): p. 123-36. 

86. Hendry SH, Reid RC (2000), The koniocellular pathway in primate vision. AnnuRev 
Neurosci, 23: p. 127-53. 

87. Hendry SH, Yoshioka T (1994), A neurochemical^ distinct third channel in the macaque 
dorsal lateral geniculate nucleus. Science, 264(5158): p. 575-7. 

88. Hubel DH (1995), Eye, brain and vision.led. New York: Scientific American Library.242. 

89. Hubel DH, Wiesel TN (1959), Receptive fields of single neurones in the cat's striate cortex. 
J Physiol, 148: p. 574-91. 

90. Hubel DH, Wiesel TN (1961), Integrative action in the cat's lateral geniculate body. 
J Physiol, 155: p. 385-98. 

91. Hubel DH, Wiesel TN (1962), Receptive fields, binocular interaction and functional archi- 
tecture in the cat's visual cortex. J Physiol, 160: p. 106-54. 

92. Hubel DH, Wiesel TN (1965), Receptive fields and functional architecture in two nonstriate 
visual areas (18 and 19) of the cat. J Neurophysiol, 28: p. 229-89. 

93. Hubel DH, Wiesel TN (1968), Receptive fields and fiinctional architecture of monkey striate 
cortex. J Physiol, 195(1): p. 215^13. 

94. Hubel DH, Wiesel TN (1972), Laminar and columnar distribution of geniculo-cortical fibers 
in the macaque monkey. J Comp Neurol, 146(4): p. 421-50. 

95. Hubel DH, Wiesel TN (1974), Sequence regularity and geometry of orientation columns in 
the monkey striate cortex. J Comp Neurol, 158(3): p. 267-93. 

96. Hupe JM, James AC, Payne BR, et al. (1998), Cortical feedback improves discrimination 
between figure and background by VI, V2 and V3 neurons. Nature, 394(6695): p. 784-7. 

97. Jones HE, Grieve KL, Wang W, Sillito AM (2001), Surround suppression in primate VI. 
J Neurophysiol, 86(4): p. 2011-28. 

98. Kandel ER, Schwartz JH, Jessell TM, eds (2000). Principles of neural science. 4th ed. 
McGraw Hill: New York. 



52 X.G. Troncoso et al. 

99. Kaneko A (1970), Physiological and morphological identification of horizontal, bipolar and 
amacrine cells in goldfish retina. J Physiol, 207(3): p. 623-33. 

100. Kapadia MK, Westheimer G, Gilbert CD (1999), Dynamics of spatial summation in primary 
visual cortex of alert monkeys. Proc Natl Acad Sci USA, 96(21): p. 12073-8. 

101. Kaplan E, Shapley RM (1982), X and Y cells in the lateral geniculate nucleus of macaque 
monkeys. J Physiol, 330: p. 125-43. 

102. Kaplan E, Shapley RM (1986), The primate retina contains two types of ganglion cells, with 
high and low contrast sensitivity. Proc Natl Acad Sci USA, 83(8): p. 2755-7. 

103. Knierim JJ, van Essen DC (1992), Neuronal responses to static texture patterns in area VI 
of the alert macaque monkey. J Neurophysiol, 67(4): p. 961-80. 

104. Kolb H, Dekorver L (1991), Midget ganglion cells of the parafovea of the human retina: a 
study by electron microscopy and serial section reconstructions. J Comp Neurol, 303(4): 
p. 617-36. 

105. Kolb H, Linberg KA, Fisher SK (1992), Neurons of the human retina: a Golgi study. J Comp 
Neurol, 318(2): p. 147-87. 

106. Kolb H, Marshak D (2003), The midget pathways of the primate retina.Doc Ophthalmol, 
106(1): p. 67-81. 

107. Lachica EA, Beck PD, Casagrande VA (1992), Parallel pathways in macaque monkey striate 
cortex: anatomically defined columns in layer III. Proc Natl Acad Sci USA, 89(8): p. 3566-70. 

108. Laubrock J, Engbert R, Kliegl R (2008), Fixational eye movements predict the perceived 
direction of ambiguous apparent motion. J Vis, 8(14): p. 1-17. 

109. Leventhal AG, Rodieck RW, Dreher B (1981), Retinal ganglion cell classes in the old world 
monkey: morphology and central projections. Science, 213(4512): p. 1 1 39—42. 

1 10. Levitt JB, Lund JS (2002), The spatial extent over which neurons in macaque striate cortex 
pool visual signals. Vis Neurosci, 19(4): p. 439-52. 

111. Livingstone M, Hubel D (1988), Segregation of form, color, movement, and depth: anatomy, 
physiology, and perception. Science, 240(4853): p. 740-9. 

112. Livingstone MS, Hubel DH (1982), Thalamic inputs to cytochrome oxidase-rich regions in 
monkey visual cortex. Proc Natl Acad Sci USA, 79(19): p. 6098-101. 

113. Livingstone MS, Hubel DH (1984), Anatomy and physiology of a color system in the primate 
visual cortex. J Neurosci, 4(1): p. 309-56. 

114. Lorente de No R (1949), Cerebral cortex: architecture, intracortical connections, motor 
projections, in Physiology of the nervous system, Fulton JF, Editor. Oxford University Press: 
Oxford, p. 288-330. 

115. Lowel S, Schmidt KE, Kim DS, et al. (1998), The layout of orientation and ocular domi- 
nance domains in area 17 of strabismic cats. Eur J Neurosci, 10(8): p. 2629^-3. 

116. Lund JS (1973), Organization of neurons in the visual cortex, area 17, of the monkey 
(Macaca mulatto). J Comp Neurol, 147(4): p. 455-96. 

117. Lund JS, Boothe RG, Lund RD (1977), Development of neurons in the visual cortex (area 
17) of the monkey (Macaca nemestrina): a Golgi study from fetal day 127 to postnatal matu- 
rity. J Comp Neurol, 176(2): p. 149-88. 

118. Lund JS, Lund RD, Hendrickson AE, et al. (1975), The origin of efferent pathways from the 
primary visual cortex, area 1 7, of the macaque monkey as shown by retrograde transport of 
horseradish peroxidase. J Comp Neurol, 164(3): p. 287-303. 

119. Lund JS, Wu CQ (1997), Local circuit neurons of macaque monkey striate cortex: IV. 
Neurons of laminae 1-3A. J Comp Neurol, 384(1): p. 109-26. 

120. Macknik SL (2006), Visual masking approaches to visual awareness. Prog Brain Res, 155: 
p. 177-215. 

121. Macknik SL, Haglund MM (1999), Optical images of visible and invisible percepts in the 
primary visual cortex of primates. Proc Natl Acad Sci USA, 96(26): p. 15208-10. 

122. Macknik SL, Livingstone MS (1998), Neuronal correlates of visibility and invisibility in the 
primate visual system. Nat Neurosci, 1(2): p. 144-9. 

123. Macknik SL, Martinez-Conde S (2004), The spatial and temporal effects of lateral inhibitory 
networks and their relevance to the visibility of spatiotemporal edges. Neurocomputing, 
58-60: p. 775-82. 



2 Vision's First Steps: Anatomy, Physiology, and Perception in the Early Visual System 53 

124. Macknik SL, Martinez-Conde S (2007), The role of feedback in visual masking and visual 
processing. Adv Cogn Psychol, 3: p. 125-52. 

125. Macknik SL, Martinez-Conde S (2009), The role of feedback in visual attention and aware- 
ness, in The Cognitive Neurosciences, 4th edition, Gazzaniga MS, Editor. MIT Press: 
Cambridege, MA, p. 1165-75. 

126. Macknik SL, Martinez-Conde S, Haglund MM (2000), The role of spatiotemporal edges in 
visibility and visual masking. Proc Natl Acad Sci USA, 97(13): p. 7556-60. 

127. Macknik SL, Martinez-Conde S (2009), Encyclopedia of Perception, Ed. E. Bruce Goldstein, 
Sage Press, 522-24. 

128. MacNeil MA, Masland RH (1998), Extreme diversity among amacrine cells: implications 
for function. Neuron, 20(5): p. 971-82. 

129. Maguire WM, Baizer JS (1984), Visuotopic organization of the prelunate gyrus in rhesus 
monkey. J Neurosci, 4(7): p. 1690-704. 

130. Malach R, Amir Y, Harel M, Grinvald A (1993), Relationship between intrinsic connections 
and functional architecture revealed by optical imaging and in vivo targeted biocytin injec- 
tions in primate striate cortex. Proc Natl Acad Sci USA, 90(22): p. 10469-73. 

131. Marks WB, Dobelle WH, Macnichol EF, Jr. (1964), Visual pigments of single primate cones. 
Science, 143: p. 1181-3. 

132. Marr D, Hildreth E (1980), Theory of edge detection. Proc R Soc Lond Series B, 207: p. 187-217. 

133. Martinez-Conde S, Cudeiro J, Grieve KL, et al. (1999), Effects of feedback projections from area 
18 layers 2/3 to area 17 layers 2/3 in the cat visual cortex. J Neurophysiol, 82(5): p. 2667-75. 

134. Martinez-Conde S, Macknik SL (2001). Junctions are the most salient visual features in the 
early visual system, in Society for Neuroscience 31st Annual Meeting. SanDiego, CA. 

135. Martinez-Conde S, Macknik SL (2007), Windows on the mind. Sci Am, 297(2): p. 56-63. 

136. Martinez-Conde S, Macknik SL (2008), Fixational eye movements across vertebrates: com- 
parative dynamics, physiology, and perception. J Vis, 8(14): p. 1-16. 

137. Martinez-Conde S, Macknik SL, Hubel DH (2000), Microsaccadic eye movements and firing 
of single cells in the striate cortex of macaque monkeys. Nature Neuroscience, 3(3): p. 251-8. 

138. Martinez-Conde S, Macknik SL, Hubel DH (2002), The function of bursts of spikes during 
visual fixation in the awake primate lateral geniculate nucleus and primary visual cortex. 
Proc Natl Acad Sci USA, 99(21): p. 13920-5. 

139. Martinez-Conde S, Macknik SL, Hubel DH (2004), The role of fixational eye movements in 
visual perception. Nat Rev Neurosci, 5: p. 229^-0. 

140. Martinez-Conde S, Macknik SL, Troncoso XG, Dyar TA (2006), Microsaccades counteract 
visual fading during fixation. Neuron, 49(2): p. 297-305. 

141. Martinez-Conde S, Macknik SL, Troncoso XG, Hubel DH (2009), Microsaccades: a neuro- 
physiological analysis. Trends Neurosci, 32(9): p. 463-75. 

142. Martinez LM, Alonso JM (2001), Construction of complex receptive fields in cat primary 
visual cortex. Neuron, 32: p. 515-25. 

143. Martinez LM, Wang Q, Reid RC, et al. (2005), Receptive field structure varies with layer in 
the primary visual cortex. Nat Neurosci, 8(3): p. 372-9. 

144. Masland RH, Ames A, 3rd (1976), Responses to acetylcholine of ganglion cells in an iso- 
lated mammalian retina. J Neurophysiol, 39(6): p. 1220-35. 

145. Maunsell JH, Newsome WT (1987), Visual processing in monkey extrastriate cortex. Annu 
Rev Neurosci, 10: p. 363^101. 

146. McGuire BA, Gilbert CD, Rivlin PK, Wiesel TN (1991), Targets of horizontal connections 
in macaque primary visual cortex. J Comp Neurol, 305(3): p. 370-92. 

147. Meadows JC (1974), The anatomical basis of prosopagnosia. J Neurol Neurosurg Psychiatry, 
37(5): p. 489-501. 

148. Meadows JC (1974), Disturbed perception of colours associated with localized cerebral 
lesions. Brain, 97(4): p. 615-32. 

149. Merigan WH, Maunsell JH (1993), How parallel are the primate visual pathways? Annu 
Rev Neurosci, 16: p. 369^102. 

150. Miller RF, Slaughter MM (1986), Excitatory amino acid receptors of the retina: diversity 
and subtype and conductive mechanisms. TINS, 9: p. 211-3. 



54 X.G. Troncoso et al. 

151. MishkinM, Ungerleider LG (1983), Object vision and spatial vision: two cortical pathways. 
Trendes Neurosci, 6: p. 414-7. 

152. Mollon JD, Bowmaker JK (1992), The spatial arrangement of cones in the primate fovea. 
Nature, 360(6405): p. 677-9. 

153. Mountcastle VB (1957), Modality and topographic properties of single neurons of cat's 
somatic sensory cortex. J Neurophysiol, 20(4): p. 408-34. 

154. Mountcastle VB, Berman AL, Davies PW (1955), Topographic organization and modality 
representation in first somatic area of cat's cerebral cortex by method of singleunit analysis. 
Am J Physiol, 183: p. 646. 

155. Movshon JA, Thompson ID, Tolhurst DJ (1978), Spatial summation in the receptive fields of 
simple cells in the cat's striate cortex. J Physiol, 283: p. 53-77. 

156. Muller JF, Dacheux RF (1997), Alpha ganglion cells of the rabbit retina lose antagonistic 
surround responsesunder dark adaptation. Vis Neurosci, 14(2): p. 395-401. 

157. Murphy PC, Duckett SG, Sillito AM (1999), Feedback connections to the lateral geniculate 
nucleus and cortical response properties. Science, 286(5444): p. 1552^4. 

158. Murphy PC, Sillito AM (1987), Corticofugal feedback influences the generation of length 
tuning in the visual pathway. Nature, 329(6141): p. 727-9. 

159. Nawy S, Copenhagen DR (1987), Multiple classes of glutamate receptor on depolarizing 
bipolar cells in retina. Nature, 325(6099): p. 56-8. 

160. Nelson R, Famiglietti EV, Jr., Kolb H (1978), Intracellular staining reveals different levels 
of stratification for on- and off-center ganglion cells in cat retina. J Neurophysiol, 41(2): 
p. 472-83. 

161. Nelson R, Kolb H (1983), Synaptic patterns and response properties of bipolar and ganglion 
cells in the cat retina. Vision Res, 23(10): p. 1183-95. 

162. Obermayer K, Blasdel GG (1993), Geometry of orientation and ocular dominance columns 
in monkey striate cortex. J Neurosci, 13(10): p. 41 14-29. 

163. Olavarria JF, Van Essen DC (1997), The global pattern of cytochrome oxidase stripes in 
visual area V2 of the macaque monkey. Cereb Cortex, 7(5): p. 395-404. 

164. 0sterberg G (1935), Topography of the layer of rods and cones in the human retina. Acta 
Ophthalmologica, 6: p. 1-103. 

165. Pack CC, Livingstone MS, Duffy KR, Born RT (2003), End-stopping and the aperture prob- 
lem: two-dimensional motion signals in macaque VI. Neuron, 39(4): p. 671-80. 

166. Pasupathy A, Connor CE (1999), Responses to contour features in macaque area V4. 
J Neurophysiol, 82(5): p. 2490-502. 

167. Pearlman AL, Birch J, Meadows JC (1979), Cerebral color blindness: an acquired defect in 
hue discrimination. Ann Neurol, 5(3): p. 253-61. 

168. Peichl L, Wassle H (1983), The structural correlate of the receptive field centre of alpha 
ganglion cells in the cat retina. J Physiol, 341: p. 309-24. 

169. Perkel DJ, Bullier J, Kennedy H (1986), Topography of the afferent connectivity of area 17 
in the macaque monkey: a double-labelling study. J Comp Neurol, 253(3): p. 374-402. 

170. Perry VH, Cowey A (1981), The morphological correlates ofX- and Y-like retinal ganglion 
cells in the retina of monkeys. Exp Brain Res, 43(2): p. 226-8. 

171. Perry VH, Oehler R, Cowey A (1984), Retinal ganglion cells that project to the dorsal lat- 
eral geniculate nucleus in the macaque monkey. Neuroscience, 12(4): p. 1 101-23. 

172. Poggio GF, Doty RW, Jr., Talbot WH (1977), Foveal striate cortex of behaving monkey: 
single-neuron responses to square-wave gratings during fixation of gaze. J Neurophysiol, 
40(6): p. 1369-91. 

173. Polyak S (1941), The retina. Chicago: University of Chicago Press. 

174. Powell TP, Mountcastle VB (1959), Some aspects of the functional organization of the cortex 
of the postcentral gyrus of the monkey: a correlation of findings obtained in a singleunit 
analysis with cytoarchitecture. Bull Johns Hopkins Hosp, 105: p. 133-62. 

175. Ramon y Cajal S (1893), La retine des vertebres. Cellule, 9: p. 117-257. 

176. Ramon y Cajal S (1900), Structure of the Mammalian Retina. Madrid. 



2 Vision's First Steps: Anatomy, Physiology, and Perception in the Early Visual System 55 

177. Rao RPN, Olshausen BA, Lewicki MS (2002), Probabilistic models of the brain: perception 
and neural function. Cambridge, MA: MIT Press. 

178. Ratcliff G, Davies-Jones GA (1972), Defective visual localization in focal brain wounds. 
Brain, 95(1): p. 49-60. 

179. Ratliff F (1965), Mach bands: Quantitative studies on neural networks in the retina. San 
Francisco: Holden-Day, Inc. 

180. Reid RC, Alonso JM (1995), Specificity of monosynaptic connections from thalamus to 
visual cortex. Nature, 378(6554): p. 281-4. 

181. Ringach DL (2002), Orientation selectivity in macaque VI: diversity and laminar depen- 
dence. J Neurosci, 22(13): p. 5639-51. 

182. Ringach DL (2002), Spatial structure and symmetry of simple cell receptive fields in 
macaque primary visual cortex. J Neurophysiol, 88: p. 455-463. 

183. Rockland KS, Lund JS (1983), Intrinsic laminar lattice connections in primate visual cortex. 
J Comp Neurol, 216(3): p. 303-18. 

184. Rockland KS, Saleem KS, Tanaka K (l994),Divergent feedback connections from areas V4 
and TEO in the macaque. Vis Neurosci, 11(3): p. 579-600. 

185. Rockland KS, Virga A (1989), Terminal arbors of individual "feedback" axons project- 
ing from area V2 to VI in the macaque monkey: a studyusing immunohistochemistry of 
anterogradely transported Phaseolus vulgaris-leucoagglutinin. J Comp Neurol, 285(1): 
p. 54-72. 

186. Rodieck RW (1998), The first steps in seeing. Sunderland, Massachusetts: Sinauer 
Associates. 562. 

187. Roorda A, Williams DR (1999), The arrangement of the three cone classes in the living 
human eye. Nature, 397(6719): p. 520-2. 

188. Sceniak MP, Hawken MJ, Shapley R (2001), Visual spatial characterization of macaque VI 
neurons. J Neurophysiol, 85(5): p. 1873-87. 

189. Schiller PH, Malpeli JG (1978), Functional specificity of lateral geniculate nucleus laminae 
of the rhesus monkey. I Neurophysiol, 41(3): p. 788-97. 

190. Schultze M (1866), Zur Anatomieund Physiologie der Retina. Arch Mikrosk Anat 
Entwicklungsmech, 2: p. 165-286. 

191. Shapley R, Hawken M, Ringach DL (2003), Dynamics of orientation selectivity in the pri- 
mary visual cortex and the importance of cortical inhibition. Neuron, 38(5): p. 689-99. 

192. Shapley R, Perry JS (1986), Cat and monkey retinal ganglion cells and their visual func- 
tional roles. Trendes Neurosci, 9: p. 229-35. 

193. Sherman SM, Guillery RW (1998), On the actions that one nerve cell can have on another: 
distinguishing "drivers" from "modulators" . Proc Natl Acad Sci USA, 95(12): p. 7121-6. 

194. Sherman SM, Guillery RW (2001), Exploring the thalamus. SanDiego: Academic Press. 

195. Shipp S, Zeki S (1985), Segregation of pathways leading from area V2 to areas V4 and V5 
of macaque monkey visual cortex. Nature, 315(6017): p. 322-5. 

196. Shipp S, Zeki S (1989), The organization of connections between areas V5 and VI in 
macaque monkey visual cortex. Eur J Neurosci, 1(4): p. 309-32. 

197. Sincich LC, Horton JC (2005), The circuitry of VI and V2: integration of color, form, and 
motion. Annu Rev Neurosci, 28: p. 303-26. 

198. Skavenski AA, Hansen RM, Steinman RM, Winterson BJ (1979), Quality of retinal image stabi- 
lization during small natural and artificial body rotations in man. Vision Res, 19(6): p. 675-83. 

199. Slaughter MM, Miller RF (1981), 2-amino-4-phosphonobutyric acid: a new pharmacologi- 
cal tool for retina research. Science, 211(4478): p. 182-5. 

200. Slaughter MM, Miller RF (1983), An excitatory amino acid antagonist blocks cone input to 
sign-conserving second-order retinal neurons. Science, 219(4589): p. 1230-2. 

201. Steriade M, Jones EG, McCormick DA, eds (1997). Thalamus. Elsevier: New York. 

202. Stone J, Dreher B, Leventhal A (1979), Hierarchical and parallel mechanisms in the 
organization of visual cortex. Brain Res, 180(3): p. 345-94. 

203. Suzuki W, Saleem KS, Tanaka K (2000), Divergent backward projections from the anterior part 
of the inferotemporal cortex (area TE) in the macaque. J Comp Neurol, 422(2): p. 206-28. 



56 X.G. Troncoso et al. 

204. Tomita T (1965), Electrophysiological study of the mechanisms subserving color coding in 
the fish retina. Cold Spring Harb Symp Quant Biol, 30: p. 559-66. 

205. Trifonov YA (1968), Study of synaptic transmission between the photoreceptor and the hori- 
zontal cellusing electrical stimulation of the retina. Bioiizika, 10: p. 673-80. 

206. Troncoso XG, Macknik SL, Martinez-Conde S (2005), Novel visual illusions related to 
Vasarely 's 'nested squares ' show that corner salience varies with corner angle. Perception, 
34(4): p. 409-20. 

207. Troncoso XG, Macknik SL, Martinez-Conde S (2008), Microsaccades counteract perceptual 
filling-in. J Vis, 8(14): p. 1-9. 

208. Troncoso XG, Macknik SL, Martinez-Conde S (2009), Corner salience varies linearly with 
corner angle during flicker-augmented contrast: a general principle of corner perception 
based on Vasarely's artworks. Spat Vis, 22(3): p. 21 1-24. 

209. Troncoso XG, Macknik SL, Otero-Millan J, Martinez-Conde S (2008), Microsaccades drive 
illusory motion in the Enigma illusion. Proc Natl Acad Sci USA, 105(41): p. 16033-8. 

210. Troncoso XG, Tse PU, Macknik SL, et al. (2007), BOLD activation varies parametrically 
with corner angle throughout human retinotopic cortex. Perception, 36(6): p. 808-20. 

211. Ts'o DY, Frostig RD, Lieke EE, Grinvald A (1990), Functional organization of primate 
visual cortex revealed by high resolution optical imaging. Science, 249(4967): p. 417-20. 

212. Ts'o DY, Gilbert CD, Wiesel TN (1986), Relationships between horizontal interactions 
and functional architecture in cat striate cortex as revealed by cross-correlation analysis. 
J Neurosci, 6(4): p. 1160-70. 

213. Ungerleider LG, Desimone R (1986), Cortical connections of visual area MT in the 
macaque. J Comp Neurol, 248(2): p. 190-222. 

214. Ungerleider LG, Desimone R (1986), Projections to the superior temporal sulcus from the 
central and peripheral field representations of VI and V2. J Comp Neurol, 248(2): p. 147-63. 

215. Ungerleider LG, Mishkin M (1982), Two cortical visual systems, in Analysis of visual 
behavior, Ingle DG, Goodale MA, Mansfield JQ, Editors. MIT Press: Cambridge, MA. 
p. 549-86. 

216. Usrey WM, Alonso JM, Reid RC (2000), Synaptic interactions between thalamic inputs to 
simple cells in cat visual cortex. J Neurosci, 20(14): p. 5461-7. 

217. vanDam LC, van Ee R (2006), Retinal image shifts, but not eye movements per se, cause 
alternations in awareness during binocular rivalry. J Vis, 6(11): p. 1172-9. 

218. Van Essen DC, Anderson CH, Felleman DJ (1992), Information processing in the primate 
visual system: an integrated systems perspective. Science, 255(5043): p. 419-23. 

219. Van Essen DC, Gallant JL (1994), Neural mechanisms of form and motion processing in the 
primate visual system. Neuron, 13(1): p. 1-10. 

220. Van Essen DC, Zeki SM (1978), The topographic organization of rhesus monkey prestriate 
cortex. J Physiol, 277: p. 193-226. 

221. Vasarely V (1970), Vasarely II, Plastic arts of the 20th century, ed. Joray M. Switzerland: 
Editions duGriffon Neuchatel. 

222. Verweij J, Dacey DM, Peterson BB, Buck SL (1999), Sensitivity and dynamics of rod signals 
in HI horizontal cells of the macaque monkey retina. Vision Res, 39(22): p. 3662-72. 

223. Wassle H, Boycott BB (1991), Functional architecture of the mammalian retina. Physiol 
Rev, 71(2): p. 447-80. 

224. Watanabe M, Rodieck RW (1989), Parasol and midget ganglion cells of the primate retina. 
J Comp Neurol, 289(3): p. 434-54. 

225. Werblin FS, Dowling JE (1969), Organization of the retina of the mudpuppy, Necturus macu- 
losus. II. Intracellular recording, i Neurophysiol, 32(3): p. 339-55. 

226. Wiesel TN, Hubel DH, Lam DM (1974), Autoradiographic demonstration of ocular- 
dominance columns in the monkey striate cortex by means of transneuronal transport. Brain 
Res, 79(2): p. 273-9. 

227. Wiser AK, Callaway EM (1996), Contributions of individual layer 6 pyramidal neurons to 
local circuitry in macaque primary visual cortex. J Neurosci, 16(8): p. 2724-39. 

228. Yarbus AL (1967), Eye movements and vision. New York: Plenum Press. 



2 Vision's First Steps: Anatomy, Physiology, and Perception in the Early Visual System 57 

229. Zeki SM (1974), Cells responding to changing image size and disparity in the cortex of the 
rhesus monkey. J Physiol, 242(3): p. 827-41. 

230. Zeki SM (1974), Functional organization of a visual area in the posterior bank of the superior 
temporal sulcus of the rhesus monkey. J Physiol, 236(3): p. 549-73. 

231. Zeki SM (1978), Functional specialisation in the visual cortex of the rhesus monkey. Nature, 
274(5670): p. 423-8. 

232. Zeki SM (1978), Uniformity and diversity of 'structure and function in rhesus monkey prestriate 
visual cortex. J Physiol, 277: p. 273-90. 

233. Zhaoping L (2005), The primary visual cortex creates a bottom-up saliency map, in 
Neurobiology of Attention, Itti L, Rees G, Tsotsos JK, Editors. Elsevier: Oxford, p. 570-75. 

234. Zihl J, von Cramon D, Mai N (1983), Selective disturbance of movement vision after bilateral 
brain damage. Brain, 106 (Pt2): p. 313-40. 



Chapter 3 

Retinal Remodeling and Visual Prosthetics 

Bryan W. Jones, Robert E. Marc, and Carl B. Watt 



Abstract Retinal degenerative disease induces a cascade of events that ultimately 
result in phased revision of neuronal populations and circuitry of the retina. These 
changes reveal plasticity in the retina that mimics that seen during development 
and in instances of neural deafferentation in other central nervous system (CNS) systems, 
involving neuronal as well as glial cell populations. These retinal remodeling 
changes occur across the spectrum of retinal degenerative disease and are observed 
in defects of the retinal pigment epithelium (RPE), rhodopsin packaging and trans- 
port defects as well as other non-retinitis pigmentosa (RP) related diseases with the 
final result being fundamental revision of neuronal populations and circuitry. These 
revisions impact potential biological and bionic rescues of visual function and must 
be overcome before vision restoration strategies can be viable. 
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3.1 Introduction 

Our understanding of retinal structure and function has been a 150 year journey 
through biological science with the goal of understanding precisely how the 
retina is anatomically composed and how that structure interacts physiologically. 
Unfortunately this goal while close to completion in some areas, remains woefully 
lacking in complete detail of development, participants, connectivity, physiology 
and pathology, particularly in disease processes. This chapter examines the retina 
in disease and will attempt to clarify some of the long misunderstood aspects of 
retinal pathology, discussing how those pathologies impact bionic and/or biological 
strategies in the rescue of vision. 

Other chapters in this book will discuss bionic approaches to "curing" vision 
loss using various prosthetics, leaving this chapter to function as a biological primer 
of sorts to introduce some of the biological realities that any therapeutic interven- 
tion will have to deal with as all of the inherited retinal degenerations studied to 
date reveal a biological moving target that must be considered prior to therapy. 

The question of whether or not the neural retina is receptive to bionic or biological 
intervention is one that historically has been investigated without consideration of 
the actual disease process neural systems proceed through when they experience 
loss of photoreceptors. Though the neural retina grossly appears to survive photo- 
receptor loss in diseases such as retinitis pigmentosa (RP) and age-related macular 
degeneration (AMD), the reality is that the retina is no different from other CNS 
pathways when their afferent inputs are lost. When photoreceptor inputs are lost, 
the retina engages in a wide variety of remodeling events driven by loss of signaling 
inputs. These transformations include glial hypertrophy and possible hyperplasia, 
neuronal translocations, neuronal loss and the emergence of retinal circuit altera- 
tion with the formation of novel synaptically active neuronal processes. These new 
processes are perhaps the most significant impediment to prosthetic retinal rescue 
through bionic or biological interventions as the disease process corrupts and modifies 
the normal visual information processing so as to make it indecipherable by the 
visual cortex. If we are to proceed, vision rescue strategies need to contend with 
the biological realities of retinal remodeling. 



3.2 Background 

The effort to build, design and implement neuroprosthetic devices has been chal- 
lenging due to not just the complexity of the retina, but also the difficulty of dealing 
with a complex, reactive biological tissue that changes its fundamental connectivity 
in disease processes. Efforts from a variety of labs now make it clear that the neural 
retina does not adopt a passive role with respect to photoreceptor degenerations and 
that extensive alterations and remodeling occur from the molecular scale up through 
the synaptic, cellular and tissue levels [2, 18, 19, 21, 31, 33, 49, 51, 52, 54, 58, 60, 
68-71, 80, 85, 86, 88-90, 92]. Regardless of whether the intervention is survival 
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factor delivery [29, 30], genetic [8, 61], cellular [36, 97, 105] or bionic [27, 46, 56, 
102, 1 10], approaches will have to address the ongoing process of neuronal death, 
alterations to gene expression, neuronal circuit rewiring and migration and the 
elaboration of novel glial barriers. While these prospects may appear daunting, 
prosthetic devices may in fact be the ideal intervention with which to rescue and 
reconfigure neural retinas altered by disease. 



3.3 Retinal Disease and Its Diversity 

Retinal disease including the well characterized retinitis pigmentosa (Fig. 3.1) with 
an incidence of 1-4,000 [11] and the less well understood, yet far more prevalent 
age-related macular degeneration affect millions of people world wide. While RP 
affects a significant portion of the working age population, AMD is far more 
common overall, with an incidence in the United States alone estimated to reach 
three million by 2020 [35]. Indeed, it has been estimated that AMD is the leading 
cause of new cases of blindness in Americans over 60 with an estimated 18% of 
Americans between 65 and 74 and 30% of Americans older than 74 showing signs 
of possible future AMD [106]. 

Regardless of the form retinal degenerative disease takes, the final common 
pathway of photoreceptor loss followed by downstream reactive biological 
processes results in a system that has proven difficult to rescue. While rescues of 




Fig. 3.1 Funduscopic image from 46 year old male with a diagnosis of X-linked retinitis pigmentosa, 
showing "pigmented bone spicules," accumulations of pigment epithelium that are formed by 
migration of the pigment epithelium into the neural retina along glial columns. These clinically 
pathologic findings are often seen in the peripheral retina in patients with RP 
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vision targeted towards the anterior eye have been possible for a great many years 
due to the accessibility and amenability of the tissues involved to pharmacological 
and surgical interventions, retinal disease presents a significantly larger challenge 
that has proven more difficult to combat due to its complex and progressive nature 
and the number of potential gene loci involved. These diverse pathological insults 
currently number close to 200 gene defects associated with various retinal diseases 
http://www.sph.uth.tmc.edu/RetNet/ including AMD, RP, diabetic retinopathy and 
glaucoma. These disease loci are located on 23 different genes in addition to mito- 
chondrial gene loci and result in vision loss through diverse mechanisms including 
defects in retinal pigment epithelium cells seen in recessive Leber congenital amau- 
rosis [1, 39, 77], defects in the ATP binding cassette transporter seen in recessive 
Stargardt disease [3-6, 17], the c-mer proto-oncogene receptor tyrosine kinase [20, 
26, 39], alterations in cilia function and intraflagellar transport [57, 104], arrestin 
[83, 84], and transducin defects [24, 107], rod cGMP phosphodiesterase defects 
[45, 74, 75], metabotropic glutamate receptor defects [25, 108], peripherin defects [16], 
fatty acid biosynthetic enzymes [96, 109], and a diverse assortment of other gene 
loci encoding proteins responsible for signaling [14, 43, 44, 101] and rhodopsin 
gene mutations [34, 47, 60]. 

Defects for AMD are likely as numerous and complex as the RP causes [5, 6, 9, 12, 
15, 17, 23, 28, 37, 41, 48, 53, 62, 76, 87, 98, 103], yet are dependent upon a number 
of potential gene defect interactions that over time and with the accumulation of other 
risk factors result in retinal degeneration of the central portion of the retina responsible 
for high acuity vision [103]. Additionally, because no one specific cause of AMD 
has been identified, there is some difficulty defining a precise definition complicated 
by significant overlap of clinical manifestations. Indeed there is even some degree of 
controversy over whether or not the pathophysiological processes responsible for 
many of the sequellae of AMD including drusen accumulation, geographic atrophy, 
pigmentary changes and alterations in the vascular network are even directly related. 
Whatever the mechanism(s) involved, the end result of AMD is likely the same; pho- 
toreceptor cell death followed by retinal remodeling. 



3.4 Retinal Remodeling 

While work prior to the last decade assumed that retinal degenerative disease only 
affected the sensory retina, it is now commonly understood that these diseases also 
involve the neural retina to dramatic fashion [2, 8, 49, 52, 69, 71, 85, 89, 91, 92]. 
The reality of retinal degenerative disease and the subsequent changes that occur to 
the anatomy and physiology of the retina present profound difficulties to prospects 
of rescue, whether that rescue is biologically based or bionic in nature. Retinas that 
have lost their principal inputs, the photoreceptors, have been effectively deaffer- 
ented and undergo changes to their circuitry early and likely initially clinically 
occult. Regardless of the initial molecular or environmental insult, the proverb 
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"omnes viae Romam ducunt" or "all roads lead to Rome" summarizes where these 
mechanisms take us with respect to retinal remodeling. All defects resulting in loss 
of photoreceptor input to the neural retina initiate a series of events that change the 
fundamental ground truth of the retinal neural circuitry. This alteration in how the 
retina processes signals presents a significant challenge to retinal rescue through 
bionic prosthetic devices or biological interventions and it can be argued that most 
approaches to intervention have waited far too long in the degenerative process to 
hope for any substantive visual rescue. By the time photoreceptors are gone 
(Fig. 3.1), the changes to wiring are well underway. 

Most implant strategies presume substantial survival of retinal outflow archi- 
tectures and while it has long been claimed that the neural retina remains 
unchanged after the death of the sensory retina, this perspective is incorrect. In 
retinal degenerations, the neural retina undergoes a series of phases initiated by a 
period of photoreceptor or retinal pigment epithelial cell stress (Fig. 3.2). The 
standard metabolic phenotypes of some cells (Miiller cells) become altered pos- 
sibly indicating fundamental changes in the abilities of these cells to maintain their 
function and viability, but neuronal metabolic profiles appear to be maintained 
until cell death. Initially clinically occult changes also occur to the circuitry of the 
neural retina as well in even early stages of retinal degeneration. Subsequent to 
phase one, the neural retina enters into a phase of outer nuclear layer modification 
that includes photoreceptor cell death, apparent death of bystander neurons, 
phagocytic consumption of dying neurons and the walling off or entombment of 
the remnant neural retina beneath Miiller cell processes. The final tertiary phase of 
retinal degeneration occurs as the retina enters a protracted period of remodeling 
characterized by disruption of topology by glial hypertrophy and continued neuronal 
migration, continued neuronal cell death and extensive rewiring with elaboration 
of de novo neurite and synaptic formation [49, 51, 52, 70]. Late in the course of 
retinal degeneration, neuronal death becomes extensive. Though many neurons 
persist after death of the sensory retina, all are susceptible to cell death in varying 
fractions and patterns. Focal depletion of the inner nuclear layer is common and 
some genetic types of photoreceptor degenerations express massive ganglion cell 
loss in large patches of retina. In the most extreme cases, the Miiller cell seal 
breaks down and neurons do in fact emigrate from the retina into the remnant 
choroid [50]. 

These three phases of retinal remodeling culminate in the rewiring of all cell 
classes and essentially reprogram the retina rendering the circuitry incapable of 
processing visual data and delivering those data to visual cortex [69]. It should be 
noted that even though the first report of aberrant circuitry in the human RP retina 
goes back to 1974 [54], the concept of neural remodeling events are abundant in the 
epilepsy literature [55, 79, 93] and the vision research community is coming late to 
the game. These alterations in retinal morphology and physiology are seen across 
the spectrum of retinal degenerations from inherited [33] to engineered [51, 70] and 
induced photoreceptor degenerations [21] with changes occurring relatively early 
after photoreceptor cell stress and death [69]. 
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Fig. 3.2 (1) Truncation of photoreceptors. (2) Rod axon extension. (3) Cone axon extension. 
(4) Rod bipolar cell dendrite retraction. (5) Cone bipolar cell dendrite retraction. (6) Horizontal 
Cell axon remodeling. (7) Glial seal. (8) Phenotype revisions. (9) Neurite fascicle formation. 
(10) Microneuroma formation. (11) Neuronal migration. (12) Neuronal death. (13) IPL rewir- 
ing. (14) Laminar deformation. A schematic representation of the three stages of retinal degen- 
eration showing both rod and cone photoreceptors, rod and cone bipolar cells, ganglion cells, a 
horizontal cell, GABAergic amacrine and glycinergic amacrine cells. The two nuclear layers are 
illustrated as horizontal bands. The first frame, native retina shows normal lamination and con- 
nectivity of cell classes in the retina. Phase 1 reveals early photoreceptor stress and outer segment 
shortening ( 1 ) along with rod and cone neurite extensions projecting down into inner nuclear 
layer and ganglion cell layer (2, 3). Horizontal cells are also seen contributing to the neurite 
projections (6) along with rod and cone bipolar cells undergoing dendrite retraction (4,5). Miiller 
cells may also begin to hypertrophy in this stage. By the end of phase 2 there is a complete loss of 
photoreceptors and elaboration of a Miiller cell seal over the neural retina (7), sealing it-off away 
from the remnant choroid. Neuronal phenotypic revisions are underway or complete at this time 
(8). Early phase 3 events ensue with the elaboration of neurite extensions from glycinergic and 
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3.5 Retinal Circuitry 

Though analysis of the neural retina and its circuitry goes back over 100 years ago 
to Ramon y Cajal's work, most work examining the anatomy of circuitry in retinal 
disease is more recent, encompassing efforts in the last three decades to understand 
the components and their function. This work has revealed the retina to be a bi- 
laminar device with sensory and computational layers. 

The sensory retina is composed of photoreceptors and is the photon transduction 
layer, while the neural retina is composed of the remaining neuron classes that 
comprise the image-processing layer. Even in a simple retina like the mammalian, 
the retinal circuitry is complex, comprising approximately 14 patterned outflow 
channels, realized as ganglion cells. The number of cell classes in the mammalian 
retina includes one rod class, one rod horizontal cell, one rod bipolar cell, two to 
three cone classes, one to three cone horizontal cells, 9+ cone bipolar cells, 27 
amacrine cells, and about 15-20 ganglion cells. Thus, about 60-70 cellular devices 
form the outflow channels [66, 73, 99]. These outflow channels involve the flow of 
information through a set of stereotypical circuits from photoreceptors to bipolar 
cells to ganglion and amacrine cells with amacrine cells providing both feedback 
and feedforward control [65, 72, 73, 99]. It should be noted however that even two 
bipolar cells providing input to two separate ganglion cells, interconnected by a 
single amacrine cell provides a combinatorial 90 distinct and separate motifs 
assuming lumped-parameter circuitry. Assuming distributed parameter circuitry 
[100] expands the number of combinatorials to over 2,000 potential motifs. This 
approximation of a circuit diagram does not include any weightings for differential 
synaptic strength, cell class diversity, and coupling by gap junctions. Nor does this 
approximation include the most common form of synaptic connection in the retina 
between cells, the amacrine-amacrine cell serial synaptic chain. However, we know 
that the outflow of signals from the mammalian retina is represented by only 15-20 
ganglion cell classes [67, 82], greatly simplifying the number of possible outputs, 
though we do not know what the total network topology is. 

Even rich models [42] that mimic physiologic data acquired over limited spatiotem- 
poral domains predict little about network topology or emergent features. Despite a 
broad view of the bounds of biophysical performance provided by physiology, models 
derived from physiology are essentially degenerate: not unique to any one network 
topology. In addition, remodeling and reprogramming of neural networks in retinal 
disease strongly argues that network scrambling is a key pathology [69]. Network motif 
diversity is analogous to genetic diversity: many connective motifs (gene sequences) are 



Fig. 3.2 (continued) GABAergic amacrine cells along with contributions from bipolar cells and 
ganglion cells forming complex tangles of processes called microneuromas (9, 10) that form 
outside the normal lamination of the inner plexiform layer, sometimes merging with the inner 
plexiform layer. These microneuromas possess active synaptic elements corruptive of normal 
signaling. By late phase 3, retinal degeneration is advanced with neuronal migration or transloca- 
tion events occurring in a bi-directional fashion (11) along with neuronal death of many cell 
classes (12). IPL rewiring and laminar deformation of the plexiform layers can also be observed 
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possible, but only a subset form good filters (proteins), and mutating motifs generates 
neural malfunction (genetic disease). 

In theory, subretinal implants drive remnant circuits with cone-like inputs and 
epiretinal implants drive ganglion cell channels by mimicking bipolar-amacrine cell 
networks. Both schemes require survival of retinal neurons to drive perceptual and 
oculomotor systems, and presume no alterations in cell patterning or connectivity, nor 
any corruptive signal invasion into retinal networks. Subretinal strategies uniquely 
require positioning within the subretinal space. These presumptions (preservation of 
topology, cell numbers and wiring) are false for most retinal degenerations. 



3.6 Retinal Circuitry Revision 

Neuronal translocations in the remodeling retina are complex and do not just 
involve migrations of cell somata that leave their dendrites and axons in the original 
locations, preserving connectivity. The reality is far more insidious as the elabora- 
tion of new neurite, axonal, and synaptic structures occurs before gross cellular 
migration ensues (Figs. 3.3 and 3.4). These structures occur individually and may 
assemble into fascicles and microneuromas that may run for many microns under- 
neath the Miiller cell seal, forever changing and corrupting completely the neuronal 
circuitry of the retina [49-52, 68-71]. Modeling of new circuits demonstrates that 
all observed circuits are corruptive and many form resonant circuits, rendering the 
remnant neural retina no longer effective as an image processor [49, 51, 69]. 

While most analyses of retinal degeneration have focused on events surrounding 
phase 2 and photoreceptor death, rewiring of the neural retina occurs in all phases 
of retinal degeneration, and likely begins prior to photoreceptor death during the 
stress phase [69, 70]. Early in phase 2, ganglion cell light responses are altered, 
resulting in the loss of ON responses with the simultaneous preservation of OFF 
responses [80]. Once the photoreceptors are completely lost, ganglion cells spike 
throughout the retina of the Pde6b ldl mouse retina [86], possibly providing a mech- 
anism behind the scintillating scotomas reported by many patients with RP [22]. 

Therefore, passive anatomy alone does not reveal the scope of neural change in 
response to retinal degenerative disease. The growing evidence supports retinal 
rewiring as a common feature in retinal degenerations that involve photoreceptor 
loss and recent work [69] indicates profound changes in physiology through the use 
of excitation mapping [63, 64] and mapping cellular identity across disease states 
with single-cell resolution [67] along with in vivo and in vitro ligand activation in 
wild-type mice and rdcl and hrhoG mutant mice exhibiting rapid photoreceptor 
degeneration. In addition, the Marc 2007 study included in vitro excitation mapping 
in a sample of human RP retina revealing reprogramming events in bipolar cells that 
likely impact all forms of proposed retinal rescue strategies as the remodeling goes 
beyond rewiring and morphological change to include molecular reprogramming. 

These findings are perhaps not surprising in that changes in circuitry have been 
documented in the literature for years. Other than the previously noted 1974 study 
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Fig. 3.3 EM image of a microneuroma underneath a distal retinal Miiller cell seal on the left with 
a blood vessel and portion of an erythrocyte on the right. One hundred to four hundred nanometer 
diameter processes are running parallel together, perpendicularly through this plane of section 
comprised of five transmission electron microscopy (TEM) images mosaiced together. Synaptic 
profiles are present in this microneuroma with apparent bipolar cell (BC) like synapses with 
dyads, yet bereft of ribbons as well as conventional synapses from amacrine cells (AC) shown in 
the inset indicating that microneuromas are potentially not passive structures with respect to 
circuitry. Efforts to reconstruct microneuromas are underway to define the pathology of circuitry 
in these retinas. Scale bars = 1 um 



by Kolb, some of the earliest indications of retinal rewiring or connectivity defects 
can be seen in a paper by Li et al. in 1995 [58] where the authors documented aber- 
rantly sprouting rod photoreceptors. Fei in 2002 [32] documented sprouting cones, 
Machida et al. [60] found abnormal sprouting of photoreceptors and horizontal 
cells in the degenerating retinas of the P23H transgenic rat and Fariss et al. [31] 
documented anomalous extension of rod, horizontal and amacrine cell neurites 
throughout the neural retina while Gregory-Evans et al. identified abnormal cone 
synapses in human cone-rod dystrophy [38]. Peng et al. identified ectopic synapses 
in the RCS rat [78] while other investigators working concurrently in mouse models 
of RP identified some of the earliest changes in the second order neurons, with 
dendritic retraction of rod bipolar and horizontal cells after photoreceptor cell loss 
[88, 90, 95]. Documentation of neuronal migration [49, 51, 52, 70], the identifica- 
tion of corruptive synaptic machinery in rod and cone bipolar cells as well as 
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Fig. 3.4 Additional synaptic structures often present in microneuromas, though with immature 
forms. This example shows a aberrant presynaptic multi-projection amacrine cell (AC) making 
synaptic contact onto a bipolar cell in parallel with another bipolar cell (BC) profile making a 
simultaneous synapse complete with synaptic ribbon, onto the same bipolar cell profile. Scale 
bar = 200 nm 



horizontal cells [18, 19] and most significantly, formation of new neuronal 
connectivities and reprogramming [69] have made for a compelling literature that 
will absolutely impact the implementation and success of rescues designed to 
preserve or restore vision. 



3.7 Implications for Bionic Rescue 



Because of the vast diversity of potential insults in both RP and AMD, any one, 
targeted intervention is likely to be useful for only a small percentage of potential 
individuals. Therefore, approaches designed to replace entire systems with bionic 
and biological solutions may appear attractive. However, as noted all retinal degen- 
erations lead to problems of access and alterations to the fundamental image 
processing circuitry. The problem of how to rescue vision is further compounded 
by the issue of when to intervene. Current therapies or interventions are limited to 
those patients who have lost a considerable portion of their vision and are legally 
blind. These patients often present at late stage with advanced retinal degeneration 
(Fig. 3.1) and already likely exhibit profound alterations to the retinal circuitry that 
corrupt any surrogate inputs. 



3 Retinal Remodeling and Visual Prosthetics 69 

While modern engineering has allowed significant advancements in miniaturization 
of circuitry combined with the ability to power potentially prosthetic devices [7, 59], 
we still are lacking in our development and implementation of visual system inter- 
faces to those devices. Additionally, the design and implementation will depend 
upon where in the visual system we intend to attempt an intervention and at what 
stage of retinal degeneration the subject might be in. Specifically, intervening in a 
degenerative retina will present an entirely different set of engineering difficulties 
than intervening at the optic nerve or the visual cortex and it could be argued that 
until we understand how each component of the visual system processes informa- 
tion, we will not be successful in the implementation of vision rescue prosthetics 
that attempt a simulation of the retinotopically organized flow of information to the 
visual cortex where properly patterned inputs result in spatiotemporally correct 
percepts. 

Furthermore, how to actually stimulate the retina is one consideration, but unless 
one knows the circuitry, or can model the circuitry, there is no predicting the 
possible output of the neural retina. Additionally, given that neurons in the retina 
appear to be relatively promiscuous with respect to contacts on cell classes they 
make during the degenerative process and that those contacts appear to be impov- 
erished, predicting the output of the retina irrespective of the type or methodology 
of stimulus will be difficult at best. Some modeling [69] predicts that circuits may 
"ring" for many seconds, essentially leaving the visual cortex no option but to filter 
these inputs out. Additionally, since retinal degeneration is a progressive disease, 
one might suspect that the neural retina will continue to remodel, possibly even 
recruiting and corrupting interventions or rescues into a continued degenerative 
process. 



3.8 Implications for Biological Rescue 

These critiques and findings can also be applied to biological rescues in that certain 
transplantation schemes may slow some forms of retinal degeneration when imple- 
mented before degeneration of the sensory retina is complete and remodeling 
becomes dominant. This however, is not a viable strategy for most human disease. 
Further, it is likely that the outcomes of most transplants will be impacted by at 
least three factors including cell fusion, improper rewiring, and co-opting of trans- 
planted cells into defective or non-functional forms by resident neurons and glia. 
Many reports of transplanted stem cells assuming phenotypes of host cells are now 
known to be instances of cell fusion [94]. It is also a distinct possibility that when 
delivery of exogenous cells induces trauma from the surgery, aberrant protein and 
DNA uptake can also alter host and guest phenotypes, confounding analysis. 

In addition to the corruptive local and global rewiring that occurs in retinal 
remodeling, retinas appear to have lost patterning restrictions as well. Naive cells 
do not carry the normally present developmental structuring that occurs during reti- 
nal maturation and do not induce re-patterning. Moreover, transplanted photoreceptors 
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or any other fragments of retina will certainly engage in wide-area neurite extensions 
if they survive, and degenerating retinas already engage in profuse generation of aberrant 
neurites. There is no evidence that any of these processes make proper connections. 

This of course also begs the question: What phenotype should an uncommitted 
stem cell assume and how will it be transcriptionally guided in forming that pheno- 
type? Additionally, properly phenotyping transplanted cells [13] is critical and most 
transplant studies fail in this regard. Any emergent phenotypes, if informed by local 
signaling from negatively remodeling cells, will most likely be co-opted into an 
aberrant phenotype. Additionally, most transplanted cells are rejected [10] or 
slowly lose their own mature phenotypes after transplantation. In short, the key 
error in transplant designs is a belief that the neural retina is normal. It is not normal 
and in the degenerate retina, there is hardly a cell type that demonstrates normality. 
The basic assumptions of transplant technologies (intactness, receptivity and 
instructional capacity of the host neural retina) are false for most retinal degenera- 
tions. Moreover, expectations that cells transplanted into negatively remodeling 
environments will restore normalcy to host cells, maintain mature phenotypes or 
assume proper phenotypes seem baseless and are, as yet, untested. 

It should be noted that biological approaches should not necessarily be thought of 
as impossible as there are a number of "lower" organisms that possess retinas far more 
complex than mammalian retinas. Yet, these organisms with more complex retinas are 
able to restore or repair to some extent damage incurred to their retinas through stem 
cell dedifferentiation and recapitulation of an approximate structure and function of the 
retina [81]. However, these appraisals are gross as there have been no efforts that these 
authors are aware of that describes the nature of the circuitry in repair zones. 



3.9 Final Remarks 

The diversity of potential defects is impressive because of the complex specializa- 
tion and highly optimized function of the mammalian retina. Because of this com- 
plexity, it may seem tempting to attempt a rescue or target a solution that would 
bypass all of the potential defects through a straightforward bionic approach, solving 
all potential blinding diseases with a single solution. Fifty years from now, this may 
in fact be how history records a cure for vision loss, but any intervention, bionic or 
otherwise is going to have to deal with a progressive disease that exhibits a plastic, 
reactive neural retina with likely downstream visual alterations in the circuitry of 
visual elements in cortex that display their own ability to adapt [40] and potentially 
remodel in response to retinal deafferentation or alteration in efferent retinal signals 
resulting from retinal rewiring. These diseases are not focal and will spread, possibly 
even involving interventions designed to rescue the retina. 
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Chapter 4 

Cortical Plasticity and Reorganization 

in Severe Vision Loss 

Eduardo Fernandez and Lotfi B. Merabet 



Abstract Blind individuals make striking adjustments to their loss of sight. Current 
experimental evidence suggests that these behavioral adaptations are based on 
dramatic neurophysiological changes at the level of the brain. In particular, is the 
fact that the occipital cortex (the area of the brain normally ascribed with visual 
processing) is functionally recruited to process non-visual sensory modalities. The 
impact of these neuroplastic changes on the success of implementing a rehabilitative 
strategy such as a visual based neuroprosthesis remains unknown. Here we discuss 
several factors such as potential limits of plasticity, potential mechanisms and meth- 
ods to modulate neuroplasticity so as to promote rehabilitative potential. We should 
thus remain aware that some of the impediments to future progress in visual neuro- 
prosthesis development are not only technical, engineering and surgical issues, but 
are also related to the development and implementation of strategies designed to inter- 
face with the visually deprived brain. New evidence regarding experience-dependent 
plasticity in the adult brain together with the achievements in other neuroprosthesis 
efforts allows cautious optimism that some degree of functional vision can be restored 
in profoundly blind individuals. However, it is essential that future research explore 
the mechanisms underlying brain plasticity following the loss of vision. These new 
findings should be integrated in order to enhance the development of suitable reha- 
bilitative strategies for each particular type of visual neuroprosthesis and achieve the 
best possible behavioral outcome for a given person using these devices. 
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SPECT Single photon emission computerized tomography 

tDCS Transcranial direct current stimulation 

TMS Transcranial magnetic stimulation 



4.1 Introduction 

The question of what happens to the brain following the loss of sight is of seminal 
importance for any rehabilitative strategy for the blind. In order to interact effec- 
tively with their environment, blind individuals have to make striking adjustments 
to their loss of sight. Growing experimental evidence now suggests that these 
behavioral adaptations are reflected by dramatic neurophysiological changes at the 
level of the brain and specifically, with regions of the brain responsible for processing 
vision itself. These changes may represent the exploitation of spatial and temporal 
processing inherent within occipital visual cortex that allow a blind individual to 
adapt to the loss of sight and remain integrated in highly visually-dependent society. 

Over the past 25 years, great strides have been made in understanding the 
neurophysiological mechanisms underlying visual perception. What is less known 
are the changes associated with how the brain adapts to the loss of sight. For 
example, what is the physiological and functional fate of cortical areas normally 
associated with the processing of visual information once vision is lost (e.g. from 
ocular disease or trauma)? Would this have an impact on the success of imple- 
menting a rehabilitative strategy such as a visual based neuroprosthesis in the hope 
of restoring functional vision? As research and development continues, we should 
be aware that some of the impediments to future progress in implementing a visual 
neuroprosthesis approach are not only technical, engineering and surgical issues, 
but are also related to the development and implementation of strategies designed 
to interface with the visually deprived brain. 

In this chapter, we will review recent advances in the knowledge about brain 
plasticity and emphasize its importance in order to achieve optimal and desired 
behavioral outcomes with respect to neuroprosthesis development. Other important 
questions that will be reviewed relate to the time course of the plastic changes and 
whether cortical areas deprived of their normal sensory input can still process the 
lost sensory modality. 



4.2 Current Concepts on Brain Plasticity and Implications 
for Visual Rehabilitation 

Classical thought has held the view that the bulk of brain development occurred 
during childhood and that thereafter there was little opportunity for dramatic adap- 
tive change. It was understood however, that adult brains must display some form 
of adaptation or "plasticity" since we are capable of sensory and motor learning 
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throughout life. Furthermore, it has been postulated that the site of these ongoing 
changes was limited to "higher order" perceptual processing areas as opposed to 
primary sensory and motor cortices. Thus, primary visual, auditory and somatosen- 
sory cortical areas were strictly implicated with seeing, hearing and touch respec- 
tively. Currently, these concepts can now be viewed as an oversimplification as 
research in the field of neuroplasticity has expanded rapidly to suggest that sensory 
modalities are not as inherently distinct and independent as was previously believed 
and that the adult brain has a remarkable capacity to change and adapt throughout 
a developmental lifetime [3, 7, 28, 34, 37, 42, 61, 63, 72, 76]. 

With respect to the discussion here, there is also considerable evidence that 
adaptive and compensatory changes occur within the brain following the loss of sight 
[12, 24, 33, 60, 61, 73, 76, 82, 83]. Current evidence suggests that in response to the 
loss of sight, regions of the occipital cortex (areas normally ascribed to the processing 
of visual information) are functionally recruited to process tactile and auditory stimuli 
and even higher order cognitive functions such as verbal memory (Fig. 4.1). 

One important question to address would be to uncover the underlying nature of 
this functional recruitment of occipital cortex to process other sensory modalities. 
Is it possible that this recruitment is related to the ability of blind subjects to extract 
greater information from the remaining sensory modalities for which they are so 
highly dependent? In this context, plasticity can be viewed as an active component 
of sensory processing capable of altering processing patterns and cortical topography. 
However, it is important to note that not all neuroplastic changes following sensory 



a Braille Reading 



b Sound Localization 




Fig. 4.1 Recruitment of occipital cortical areas in the blind in response to different tasks, (a) Braille 
reading, (b) Sound localization, (c) A verbal memory task. See text for references and more details 
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loss should be assumed to be beneficial or necessarily lead to functional recovery. 
In reality, neuroplasticity can be viewed as both a positive and negative response. 
On one hand, it can contribute to changes that are functionally adaptive when 
a sensory modality is lost. On the other, neuroplasticity can also constrain the 
degree of adaptation. Therefore, the consequences of neuroplastic change need to 
be considered not only as a consequence of sensory loss, but also with respect to an 
individual's own experiences. 

Plasticity is not only essential to allow the brain to adjust to its ever-changing 
sensory environment and experiences and improving perceptual skills, but also 
plays a crucial role in the recovery from damage and insult. This is also true with 
regard to the visual system, the adaptation to blindness and ultimately, the restora- 
tion of sight. In the case of restoring sight through neuroprosthetic means, it would 
be a great over-simplification to believe that re-introduction of the lost sensory 
input by itself will immediately restore the lost sense. Specific strategies have to be 
developed to modulate information processing by the brain and to extract relevant 
and functionally meaningful information from neuroprosthetic inputs. 



4.3 Clinical Evidence for Reorganization of Cortical 
Networks in the Blind and Visually Impaired 

It would seem reasonable to presume that in the setting of visual deprivation, the 
brain would reorganize itself to exploit the sensory inputs at its disposal [5, 17, 38, 
40, 54, 61] and in fact, the loss of sight has been associated with superior non- visual 
perception in the blind, such as auditory and tactile abilities, and even in higher 
cognitive functions such as linguistic processing and verbal memory [4, 5,12, 60, 
79, 81, 92]. These adjustments not only implicate changes in the remaining sensory 
modalities (for example, touch and hearing) but also involve those parts of the brain 
once dedicated to the task of vision itself. 

For example, the ability to read Braille is associated with an enlargement of the 
somatosensory cortical representation of the reading index finger (but not the corre- 
sponding non-reading finger) in association with the recruitment of the occipital 
visual cortex for the processing of tactile information. The functional significance 
of this cross-modal plasticity is supported by a variety of additional converging 
data. For example, Uhl et al. using event-related electroencephalography and single 
photon emission computerized tomography (SPECT) [90, 91] demonstrated that 
the primary visual areas are activated in early-blind subjects while performing a 
Braille reading task. Pascual-Leone and Hamilton had the opportunity of studying 
a congenitally blind woman who was an extremely proficient Braille reader (working 
as an editor for a Braille newsletter) who became suddenly Braille alexic while 
otherwise remaining neurologically intact, following a bilateral occipital stroke 
[40]. The interesting finding was contrary to expectations as the lesion did not 
affect the somatosensory cortex, but rather damaged the occipital pole bilaterally. 
Although she was well aware of the presence of the dot elements contained in the 
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Braille text, she was unable to extract enough information to determine the meaning 
contained in the dot patterns. Further support demonstrating the functional role of 
occipital cortex comes from the fact that reversible disruption of occipital cortex 
function, for example by transcranial magnetic stimulation (TMS), impairs Braille 
reading ability in the blind [24]. 

While initial work focused on the task of Braille reading [82, 83], increasing 
evidence has also demonstrated activation of occipital cortical areas in congenitally 
blind subjects during tasks of auditory localization [51, 93]. This issue was further 
addressed by assessing language-related brain activity implicated in speech processing 
and auditory verb-generation. It has been shown that speech comprehension activates 
not only parts of the brain associated with language (as with sighted adult controls) 
[80], but also striate and extra-striate regions of the visual cortex [5]. As with tactile 
processing, reversible functional disruption of the occipital cortex by TMS impairs 
verb-general performance only in blind subjects [4] providing further evidence that 
the recruitment of the occipital cortex in high-level cognitive processing is func- 
tionally relevant. 

The work demonstrating the functional recruitment of occipital cortex for the 
processing of non-visual information may reveal only the "tip of the iceberg" in 
terms of the brain reorganization that follows visual deprivation. Nevertheless 
these neuroplastic changes define a specific time window for the success of any 
visual neuroprosthesis (before full cross-modal adaptation) that probably is influ- 
enced by factors such as the onset and duration of visual deprivation and the 
mechanisms and profile of the visual loss (Fig. 4.2). 



Visuai Prosthesis 
implantation 




Fig. 4.2 Adaptive and compensatory changes at the occipital cortex after visual deprivation, 
(a) Following visual deafferentation, inputs from other sensory processing areas reach the 
occipital cortex via connections through multisensory cortical areas (and possible through 
direct connections). These adaptive changes include the functional recruitment of visual cortical 
areas for the processing of non-visual information such as tactile information, auditory infor- 
mation and higher-order cognitive functions (e.g. verbal memory), (b) Over time, these neuro- 
plastic changes may eventually lead to the establishment of new connections and functional 
roles with clear implications on the right time of implantation and the likelihood of success in 
recreating functional vision with a visual neuroprosthetic device 
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4.4 What Are the Limits to This Cortical Plasticity? 

It is important to clarify that evidence of cortical reorganization exists in both 
congenital and late blind individuals. However, the neuroplastic changes associated 
with partial blindness remain less understood [19, 22, 27, 58]. For example, Baker 
et al. [8] found evidence of cortical reorganization in individuals with partial visual 
field loss due to age-related macular degeneration (AMD). Dilks et al. [27] reported 
converging behavioral and neuroimaging evidence from a stroke patient consistent 
with reorganization after deafferentation of human visual cortex, but other studies have 
been unable to demonstrate the same results [58, 86]. It is also worth noting, if not at 
least anecdotally, that some blind or severely visually deprived subjects do not adapt 
well to the loss of sight. In these subjects, preliminary studies suggest that the occipital 
cortex is not or at most, only partially reorganized [20, 33, 34, 61, 62, 77]. For these 
individuals, a neuroprosthetic approach might be a functionally desirable solution but 
it is important to understand their capacity for processing visual information. 

The variability in both the behavioral and neurophysiological adaptations could 
be related not only to the profoundness of the visual loss but also the amount of 
time spent visually deprived. Thus, the time course associated with neuroplastic 
changes and whether or not cortical areas can still process visual information after 
longstanding visual deprivation are important questions to be considered. In this 
context, we have recently reported the case of a late blind patient (at the time of 
study, he had no light perception in either eye for at least 12 years), who suddenly 
experienced elementary and complex visual hallucinations located in his right 
visual field [2]. Neuro-radiological examination revealed that the patient had 
an arteriovenous malformation located in his left striate cortex. Although the under- 
lying pathophysiology of visual hallucinations in this patient is uncertain, the 
presence of visual hallucinations could represent an increased level of excitability, 
possibly related to cortical "release" [23, 57] or from an "irritative" phenomenon 
reflecting the pathological activation of neural ensembles in the regions where the 
occipital lesion is located [6]. Nonetheless, these findings provide evidence that 
the occipital cortex of some blind subjects can still generate visual perceptions and 
strengthen the notion that long-term deafferented occipital cortex can remain func- 
tionally active and be potentially recruited to process visual information despite 
the complete absence of visual input. Of course, it will be very useful to know in 
advance the feasibility of generating visual perceptions in blind subjects before the 
implantation of any visual neuroprosthetic device. In this context it has been proposed 
that image-guided transcranial magnetic stimulation (TMS), can be used as a non- 
invasive method to systematically map the visual sensations induced by focal 
stimulation of the human occipital cortex and there are already clinical protocols 
which allow to relate the localization of the real site of stimulation to characteristic 
positions in the visual field [32]. This procedure has the potential to improve our 
understanding of physiologic organization and plastic changes in the human visual 
system and to establish the degree and extent of remaining functional visual cortex 
in blind subjects (Fig. 4.3). 
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Fig. 4.3 Frameless interface to aid in the positioning of the transcranial magnetic stimulator coil 
over a subject's brain. This technique enables use of anatomical information as an interactive 
navigational guide for stimulator coil position and allows recording of the position and orientation 
of the coil at the instant of stimulation for later correlation with the response data, (a) An example of 
the live display, (b) Interface to identify the position of the TMS coil over the subject's MR1 data 
at several customized locations (modified from [32]). (c) Customized interface to facilitate the 
recording of phosphene data 

Evidence of cortical reorganization within a time frame as short as months has 
been observed in animal studies [47, 48]. However human research on this matter 
remains equivocal [8, 27] and more conclusive data and measures of these functional 
changes need to be obtained. Furthermore, although rapid reorganization seems 
possible, it may be dependent on the interaction of several factors (i.e. age, time 
since onset of blindness, disease severity, etc.). At the same time, the cerebral orga- 
nization between two given blind persons may be substantially different depending 
on the pathology, the age of blindness, personal experience, etc. Clearly, much more 
evidence is needed about the extent of cortical reorganization in the adult human 
visual system and the conditions under which reorganization occurs [26]. 



4.5 Possible Mechanisms Behind Brain Plasticity 



As stated previously, brain plasticity refers to the neurophysiological changes that 
occur in relation to the organization of the brain and in response to alterations of 
neural activity and sensory experience. These changes are especially evident during 
development and after neurologic injury and can be best conceptualized with 
encompassing multiple levels of operation from molecular to cellular levels on to 
neural systems and ultimately, on to behavior. 
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As it was first suggested by Santiago Ramon y Cajal in his Textura del Sistema 
Nervioso del Hombre y los Vertebrados [18], plasticity can be regarded as a type of 
"structural modification that implies the formation of new neural pathways through 
ramification and progressive growth of the dendritic terminals". In general, these 
changes can involve rapid reinforcement of pre-established synaptic pathways and 
even the formation of new neuronal pathways [16, 24, 37, 55, 72, 73]. Consequently, 
and as a result of experience neurotransmitters are modulated, synapses change 
their morphology, dendrites and spines grow and contract, axons change their 
trajectory and cortical representational maps are altered. 

Although several mechanisms are likely to be involved in all these processes, the 
current working model emphasizes the selective strengthening of synapses following 
classical Hebbian learning rules that have guided much of the work in both the field 
of cortical synaptic plasticity and cortical representational reorganization [41]. At 
the cellular level, it has been suggested that functional reorganization may rely on 
mechanisms involving modifications in the excitatory/inhibitory neurotransmission 
[30, 45, 95] and/or a increased synthesis of neurotrophic factors [46, 49]. In this 
way, functional reorganization correlates in time with dendritic sprouting and with 
changes in the excitatory/inhibitory neurotransmission. 

There also is evidence that the mechanisms involved in synaptic plasticity can 
vary between cortical regions [84, 85] and involve glial cells [13]. Thus, the last few 
years have provided extraordinary evidence regarding the molecular mechanisms 
underlying neuroplastic changes. The exciting future in this context involves the 
possibility of developing new approaches such as specific rehabilitation strategies 
and pharmacological interventions to modulate these processes and ultimately optimize 
rehabilitative outcomes. 



4.6 Modulation of Brain Plasticity: Recent Developments 

Enhancement of function-enabling plasticity and prevention of function-disabling 
plasticity can be accomplished through several approaches such as specific reha- 
bilitation procedures, pharmacological interventions and/or exogenous electrical/ 
magnetic stimulation. For example, a number of studies have demonstrated that 
cortical maps can be contracted or expanded by loss of peripheral inputs or by 
enhanced use [3, 9, 21, 29, 37, 55, 56, 72, 78, 96]. Furthermore, as demonstrated 
by mapping studies after micro-infarcts, it is clear that behavior is one of the most 
powerful modulators of post-injury recovery and therefore, behavioral interven- 
tion to enhance recovery is becoming increasingly popular [3, 70, 72, 75]. These 
rehabilitation therapies have significantly improved the quality of life of many of 
patients after brain damage and suggest that this ability of the brain to reorganize 
itself by experience-dependent neural plasticity could be also used to develop 
new training strategies to accelerate learning and maximize the adaptation to 
prosthetic vision devices. Learning electrically stimulated visual patterns can be 
a new and difficult experience. Although the continued use of the visual implant 
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may, by itself, restore some visual perception, carefully guided visual rehabilitation 
may be necessary to maximize the adaptation and get the most from these devices. 

There are other possible methods to modulate plasticity that are currently being 
evaluated and could potentially act as adjuvant therapy. Pharmacological approaches 
[43], for example, the coupling of D-amphetamine (d-AMPH) and rehabilitative 
training seems to be useful in promoting behavioral function as well as neurotrophic 
and neuroplastic responses in animal studies [10, 88] and these effects have also 
been established during language learning in the human [15]. However, clinical studies 
on the use of d-AMPH as a pharmacological adjunct to post-stroke rehabilitation 
have yielded mixed results probably because of type, dosage and timing of drug 
delivery [87, 94]. Nonetheless, it seems that practice-dependent changes of cortical 
plasticity can be facilitated by pharmacological strategies that rely on adrenergic and 
cholinergic mechanisms and show a trend towards decrease of antagonists to these 
neurotransmitter systems [43]. These findings provide preliminary evidence that the 
pharmacological approach combined with appropriate rehabilitation strategies may 
be beneficial to promote post-injury recovery facilitating the "re-learning" processes 
and could potentially extend to the case of visual rehabilitation. 

Another possibility to enhance training effects is the use of intracortical micro- 
stimulation techniques [1, 68, 69, 71, 75]. This procedure appears to induce den- 
dritic growth in a frequency specific manner, however direct cortical microstimulation 
requires extensive neurosurgical procedures (e.g. a craniotomy) thus questioning its 
overall feasibility within the clinical setting. Alternative approaches that could 
induce similar effects and that are being currently tested include noninvasive brain 
stimulation techniques such as transcranial magnetic stimulation (TMS) and tran- 
scranial direct current stimulation (tDCS). TMS is a non-invasive and painfree 
technique for cortical stimulation in humans [11] that consists of a magnetic field 
emanating from a wire coil held outside the head. tDCS consists of applying a weak 
electrical current (1-2 mA) to modulate the activity of neurons. Both techniques are 
able to induce electrical currents in nearby regions of the brain that can influence 
brain plasticity and reorganization [25, 44, 89]. These techniques have recently 
been tested in studies with stroke patients with very promising results [3, 66] and 
show potential for future application in the rehabilitation of persons with a visual 
neuroprosthesis. For example, high frequency repetitive transcranial magnetic 
stimulation (rTMS) over the occipital cortex could be used prior to the training sessions 
with any visual prosthetic device to enhance excitability and facilitate the interpre- 
tation of the visual percepts. Such as strategy may be especially appropriate during 
early stages of training and learning. 



4.7 Neuroplasticity and Other Neuroprostheses Efforts 

In terms of restoring lost sensory function, cochlear implant research has arguably 
been the most successful and has translated into a viable therapeutic option for over 
110,000 deaf adults and children worldwide [14, 31, 53, 59, 74]. After some time, 
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Fig. 4.4 The neural plasticity of the visual cortex can contribute to ever-improving correlation 
between the physical world and evoked phosphenes. Immediately after implantation the evoked 
phosphenes are likely to induce a poor perception of an object (the letter "E" in this example). 
However, appropriate learning and rehabilitation strategies will contribute to provide concordant 
perceptions (modified from [67]) 



and with adequate and intensive training, deaf individuals can learn to comprehend 
and in some cases, even acquire speech. By analogy, after the surgical implantation 
of any visual prosthesis, the right learning and rehabilitation strategies could poten- 
tially help to modulate the plasticity of the brain and contribute to ever-improving 
performance and more concordant perceptions [67] (Fig. 4.4). 

It is remarkable that studies investigating neuroplasticity in the deaf and following 
cochlear implantation bear striking parallels with those seen in visual neuropros 
theses development. For example, similar to activation of the occipital cortex by 
auditory stimuli in the blind, the auditory cortex (Brodmann's areas 41, 42 and 22) 
is activated in deaf subjects in response to visual stimuli [36]. Thus, like in the case 
of the blind, the removal of one sensory modality leads to neural reorganization of 
the remaining ones. Visual-auditory plasticity in the deaf has also been found 
in patients with cochlear implants [31, 50, 64, 65]. Neuroimaging studies indicate 
that after implantation of a cochlear device, primary auditory cortex is activated by 
the sound of spoken words in deaf patients who had lost hearing before the devel- 
opment of language. Interestingly, it appears that if metabolism in the auditory 
cortex is restored by cross-modal plasticity changes before implantation, the audi- 
tory cortex can no longer respond to signals from a cochlear implant installed 
afterwards and patients do not show improvement in language capabilities [52]. 
It is important to note that many changes are invariably linked to chronic electrical 
stimulation that in turn, complicate the interpretation of the neuroplastic changes 
that ensue. Again, drawing from cochlear research experience, chronic stimulation 
can lead to a significant reduction in spiral ganglion neurons; de-myelination of 
residual ganglion neurons; shrinkage of the perikaryon of neurons throughout 
the auditory pathway and reduced spontaneous activity throughout the auditory 
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pathway [31]. These neuroanatomical and physiological changes also need to be 
taken into consideration in terms of the effects of chronic electrical stimulation and 
the potential long term benefits. 



4.8 A Look at What Is Ahead 

The lessons to be learned are that simple re-introduction of the lost sensory input 
by itself might not be sufficient to restore the lost sense. For restoring functional 
vision in the blind, we must first understand how the brain adapts to blindness and 
uncover adaptive resources such as cross-modal representations. There is no doubt 
that plasticity will contribute to the success of any visual neuroprostheses, but spe- 
cific strategies will then be necessary to modulate information processing by the 
brain and to extract relevant and functionally meaningful information from the 
electrical stimulation patterns [33, 61, 62] (Fig. 4.5). 
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Fig. 4.5 Some possible experimental strategies proposed to enhance functional vision and the 
adaptation to a visual neuroprosthetic device. It should be taken into account that the rehabilitation 
of the blind is a very complex problem, requiring intimate collaborations among clinicians, basic 
scientists, engineers, educators and rehabilitative experts 
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Several studies have highlighted that following the loss of vision the brain 
undergoes profound neuroplastic changes. This plasticity takes place at a variety of 
levels, from the synaptic interactions among single neurons and the circuits in 
which neurons interact, to large-scale systems comprising those circuits. Furthermore 
it has been also suggested that glial cells could have central roles in the adaptation 
to blindness [13]. The precise understanding of these changes will be crucial in 
developing and projecting the success of novel visual neuroprosthetic strategies 
will certainly have implications for rehabilitative training and device development. 
This endeavor will require strong interactions between basic scientists, clinicians, 
engineers and rehabilitation experts to help make decisions about (a) whether 
potentially residual capacity for vision exists; (b) how this plasticity can be driven 
and (c) what the inputs should be to maximize this restitution. These issues are 
central to the development of any visual neuroprosthesis approach and will provide 
a mechanistic rationale for understanding therapeutic interventions and teaching 
strategies for the blind. 

New evidence about experience-dependent plasticity of the adult brain together 
with the achievements of other neuroprosthesis efforts allows cautious optimism 
about the possibility to restore some functional vision to profoundly blind individuals, 
but there are still several important issues that should be taken into account. Case 
studies of surgical sight restoration following long-term visual deprivation [35, 39] 
provide a relevant insight. For example, patients blinded for many years experience 
profound difficulty in various visual tasks, particularly those requiring the identifi- 
cation and recognition of objects following ocular surgical procedures aimed at 
regaining some degree of functional vision. Interestingly, if these patients were 
allowed to explore the same object through touch, they can recognize it immedi- 
ately as to register their newly acquired visual percepts with their existing senses. 
These results suggest that the simple restoration of the lost sensory input may not 
itself suffice for achieving a functional sense. One possibility to overcome this 
problem might be to develop a patient controlled system that coordinates and 
registers the visual perceptions generated by a visual prosthesis with the identifica- 
tion of objects perceived through other senses (such as touch and audition). Patients 
could then learn to integrate these concordant sources of sensory stimuli into mean- 
ingful percepts [61]. 

Finally, although the effects of neural plasticity are prominent in the context 
of any visual neuroprosthesis, they are usually unrecognized or greatly underes- 
timated. Therefore, it is essential that future research explore the mechanisms that 
underlie brain plasticity following the loss of vision and that research studies in 
the field of visual prosthesis learn to integrate these new findings to enhance the 
translation of this knowledge to clinical research and practice. We have now an 
unprecedented number of tools for the restoration of sight through artificial means 
but we have to use these tools to select appropriate candidates for implantation, to 
develop suitable rehabilitative strategies for each particular type of visual neuro- 
prosthesis and to achieve the best possible behavioral outcome for a given person 
using these devices. 
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Chapter 5 

Visual Perceptual Effects of Long-Standing 

Vision Loss 



Ava K. Bittner and Janet S. Sunness 



Abstract This chapter focuses on the changes in vision experienced by patients 
with RP and AMD. The specific aspects of vision that are reviewed include progres- 
sive changes in central acuity, contrast sensitivity, visual field, color vision, night 
vision, glare, and light and dark adaptation. Emphasis is on patients' perspectives, 
including the impact on functioning and performance of activities of daily living, as 
well as rates, patterns of vision loss, and day-to-day visual fluctuations experienced 
by those with retinal degenerative diseases. Several types of visual phenomena are 
presented, including Charles Bonnet Syndrome hallucinations in AMD, perceptual 
completion or filling-in of scotomas in AMD, remapping visual cortex in AMD, the 
preferred retinal locus in AMD, and photopsias or light show type flashes in RP. 
The proposed implications of these visual changes and phenomena as they apply to 
retinal prosthetic vision are discussed. 



Abbreviations 

AIBSE Acute idiopathic blind spot enlargement 

AMD Age-related macular degeneration 

AZOOR Acute zonal occult outer retinopathy 

CBS Charles Bonnet syndrome 

fMRI Functional magnetic resonance imaging 

GA Geographic atrophy 

MEWDS Multiple evanescent white dot syndrome 

PIC Punctate inner choroidopathy 

PRL Preferred retinal locus 

RP Retinitis pigmentosa 

VEGF Vascular endothelial growth factor 
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5.1 Introduction 

Retinal degenerations are characterized by a loss of vision. The loss of photoreceptors 
leads to the development of blind spots (scotomas) or reduction in the visual field 
area. The features of the scotomas (whether they are peripheral or central, the time 
course of development, etc) are characteristic of the particular retinal disease, but 
all have in common the loss of vision. 

However, as a consequence of the retinal disease, whether at the level of "sick" 
retinal cells, changes in the optic nerve, or changes in the brain, there is the generation 
of new visual phenomena. Patients may report flashing lights (photopsias), positive 
scotomas (perceived as blurry or missing areas of vision), filling-in phenomena, 
and visual hallucinations. While the basis of these phenomena is not clearly under- 
stood, they are reported by a large number of patients, and must be taken into 
account both in the design of visual prostheses and when interpreting visual 
responses from patients implanted with such devices. 



5.2 Vision Changes Experienced by RP Patients 
5.2.1 Overview 

The most prominent and earliest symptoms of RP are progressive night blindness 
and field loss, though central vision may also be reduced. The vision loss is bilat- 
eral and symmetrical. There are two patterns of night vision loss [35]. In type 1 
rod-cone degeneration, there is reduced night vision from birth. In these patients, 
early rod dysfunction may be demonstrated by dark-adapted two-color static perimetry. 
In type 2 (sometimes called regional), night vision is normal until field loss 
begins. In type 2 patients, dark-adapted visual field perimetry shows rod photore- 
ceptor function in non-scotomatous retinal areas. Usually the patients with type 2 
degeneration once had the ability to see stars at night, while patients with type 1 
were never able to see stars. In both forms, however, the initial symptoms typically 
include either mobility problems in dim or dark illumination or an inability to 
change quickly from one light level to another. 

Most individuals are first symptomatic between the ages of 5-30 years, 
although some cases have been reported to emerge later in life [39]. The age of 
onset of RP varies for different genetic mutations, but across all patients, the aver- 
age age at which RP was diagnosed by an ophthalmologist was reported as 35 
years [56]. As a generalization, patients with X-linked RP begin having visual field 
loss the earliest (typically during the teenage years), patients with autosomal reces- 
sive RP are in the middle, and patients with autosomal dominant RP may not 
develop significant field loss until the 40s or later. The proportion of RP patients 
with autosomal recessive inheritance is approximately 30^-0%, autosomal domi- 
nant is presumed to occur in about 50-60%, and x-linked inheritance is estimated in 
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5-15% [22]. For patients without a prior family history of RP, it is often diagnosed 
incidentally during a routine eye examination, and sometimes on the basis of sub- 
jectively reported reduced night vision. 



5.2.2 Visual Field Loss in RP 

Peripheral visual field loss is universal in RP. It typically starts in the midperipheral 
region of the retina and spreads both out toward the periphery and in toward the 
macula [39]. Patients may develop a full or partial mid-peripheral ring scotoma, 
which then expands outward and inward. Since the nasal retina extends more 
peripherally than the temporal retina, the far periphery of the temporal visual field 
may be spared when the scotoma reaches the edge of the nasal, superior and inferior 
fields. Some RP patients may retain far peripheral, temporal islands of vision later 
in the course of the degeneration, even when the central field is <20° or in some 
cases when there is no remaining central vision. If the peripheral spared areas are 
large enough, they can enable patients to detect moving objects from the side or 
give valuable information during mobility to avoid bumping into objects or people. 
Early in the course of the disease, individuals with RP may be labeled as being 
clumsy or careless in terms of mobility, bumping into people and obstacles hidden 
by their (as yet unknown) scotomas. As peripheral visual field loss progresses, they 
are increasingly prone to bumps, bruises and falls. 

The rate of visual field progression in RP is typically slow, with estimates of 
about 5-14% lost per year [5, 20, 27, 34]. Figures 5.1 and 5.2 show examples 
of visual field progression in retinitis pigmentosa measured by Goldmann perimetry 
over 13 and 16 years, respectively [21]. For most individuals, the progression is 
steady, but some report that their rate of visual field loss is variable over time, with 
occasional lengthy periods of perceived stabilization. The slow rate of progression 
enables RP patients to adapt well to their vision loss, and they often do not seek 
mobility training or assistance until late in the disease when only a few degrees of 
central vision remain. A previous survey indicated that about 23% of RP patients 
were not aware that they had visual field loss, although they showed constriction of 
their field [24]. Often patients who have good central acuity but substantial field 
loss, who would be characterized as legally blind on the basis of a visual field 
diameter <20°, are surprised to learn the extent of the loss through visual field testing. 
This is because they have adjusted gradually to the progressive field loss and are 
still fully functional in their daily activities. Due to constrictions in the visual field, 
RP patients tend to use scanning to survey their environment for orientation and 
mobility. When walking in unfamiliar areas, instead of gazing straight ahead at a 
distant target, RP patients tend to direct their gaze at nearby objects on the walls, 
downward, or at the layout (i.e., edge-lines or boundaries between walls) [57]. The 
smaller the horizontal visual field extent, the more they tend to use downward- 
directed fixations, which are important to detect changes in the walking surface and 
avoid low-lying obstacles. 
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Fig. 5.2 (a-e) Goldmann visual fields obtained on the right eye of a patient with retinitis pigmentosa, 
with targets as marked, showing a pattern 1IB loss of visual fields over a period of 16 years. 
Reprinted from [21], with permission 



5.2.3 Changes in Color Vision and Glare Sensitivity in RP 



Early in RP, color vision is typically normal since the central visual field where the 
vast majority of cone photoreceptors are located is not affected by the initial rod 
photoreceptor degeneration. However, as the disease progresses, abnormalities in 
color vision are highly correlated with the extent of visual field loss. Among those 
with a visual acuity of 20/30 or better, autosomal dominant cases are less likely to 
show extensive color defects when compared to other genetic types of RP [15]. As 
central visual acuity is initially lost, the development of dyschromatopsia to pale, 
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desaturated and similar colors occurs first. Then, defects in blue color discrimination 
are common in RP. The color matches of RP patients, are on average more prota- 
nomalous (i.e., requiring a greater than normal red/green mixture ratio during color 
matching) than those in normally sighted individuals [60] . As the retinal degeneration 
progresses, bright red and orange colors are the last hues that are typically lost. 

Most RP patients become increasingly sensitive to light, which can include 
bright sunlight or diffuse glare, as in white cloudy weather. Many RP patients com- 
plain of visual impairment or of discomfort in bright light, independent of cataract. 
The amount of intraocular light scatter in RP has been correlated to visual field area 
[2]. It is possible that when vision is reduced due to the retinal degeneration, even 
a minimal further reduction due to bright light may move the patient into a range 
of functional disability [18]. Also, RP patients require a longer time to recover 
visual acuity following transitions between areas with different levels of light. 
Therefore, they experience difficulty when transitioning from a bright sunny day 
outdoors to dimmer indoor lighting, or vice versa [26]. To help with light sensitivity 
and glare, the majority of RP patients always wear sunglasses or tinted lenses when 
outdoors on sunny days. Most wear them only sometimes on cloudy days, and 
rarely if ever indoors [58]. 



5.2.4 Vision Fluctuations in RP 

There are visual phenomena present in RP patients that are unexplained at present. 
The most striking is the presence of "good" and "bad" days. Many patients report 
having good and bad days, without any clear correlation with ambient lighting or 
weather. Visual acuity and contrast sensitivity measures are two to three times more 
variable in legally blind RP patients when compared to normally-sighted individuals 
[31]. Variability in visual acuity or visual field appears to increase as visual acuity 
or visual field is reduced in RP; however, contrast sensitivity does not appear to 
vary according to the level of remaining contrast sensitivity. 

Periodic shifts or changes in the way the retinal degeneration affects patients' 
ability to function and accomplish important tasks leads to experiences of increased 
disability at potentially critical times. Some RP patients indicate that stress or fatigue 
decreases vision temporarily, and that their vision improves when these factors are 
alleviated. Day-to-day decreases in visual field test results appear to be related to 
corresponding periodic increases in perceived stress or decreases in general health. 
Research in this area, based on patient feedback and focused on investigating the 
concerns pertinent to patients, is currently being conducted to understand and miti- 
gate RP patients' visual fluctuations that can result in significant distress, morbidity, 
and reduced quality of life [32]. Attempts to understand and manage these aspects 
of retinal disease processes may also help identify therapies and improve the reli- 
ability of outcome measures in clinical trials. Day-to-day fluctuations in retinal 
sensitivity in response to electrical stimulation with prostheses and resulting short- 
term variations in visual function are also likely to occur with retinal prostheses. 
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5.3 Visual Changes in Patients with Advanced 
Macular Degeneration 

Unlike RP, patients with macular disease experience visual loss within the central 
field of vision, and generally have preserved peripheral vision throughout the 
course of the disease. The most common type of advanced macular degeneration 
is that associated with age-related macular degeneration (AMD). Two types of 
advanced AMD lead to loss of central vision, and are commonly referred to as the 
wet and dry forms of AMD. In patients with advanced AMD, about two-thirds 
have the wet form. 



5.3.1 Changes Due to Wet AMD or Choroidal 
Neovascularization 

Wet AMD (or choroidal neovascularization) is characterized by the growth of 
abnormal new blood vessels underneath the retina, with a predilection for devel- 
opment in the foveal region (i.e. the very center of the vision, specialized and 
required for fine vision such as reading). These new blood vessels leak, bleed, 
and scar, leading to an acute drop of vision and severe visual acuity loss with 
progression [7]. Despite a variety of treatments (including various types of laser 
and photodynamic therapy), until 2005 most patients with wet AMD eventually 
lost vision to the 20/200 or worse level, because of recurrence of the new blood 
vessels. Only in this decade have there been treatments developed that address the 
root cause of the development of new blood vessels. Vascular endothelial growth 
factor (VEGF) has been identified as one of the factors stimulating new blood 
vessel growth. Several anti-VEGF medications have been developed. These 
require injection into the eye, on a monthly basis. Clinical trials using 
Ranibizumab (Lucentis) have shown that about 90% of patients achieve stabiliza- 
tion of their visual acuity, and 35% improve their visual acuity [42]. Thus, while 
wet AMD has been the leading cause of severe visual loss in the population over 
age 60, the frequency of severe visual loss from this condition should decrease 
markedly in the future. Patients may still be left with scotomas, with features 
similar to those described below. 



5.3.2 Changes Due to Dry AMD or Geographic Atrophy 

The second type of advanced AMD is known as the dry AMD form or geographic 
atrophy (GA). In this condition, there is gradual death of the retinal pigment epi- 
thelium, the layer of cells beneath the retina, with consequent death of the overlying 
photoreceptors and scotoma development. These areas of atrophy and scotoma may 
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Fig. 5.3 Progression of advanced dry age-related macular degeneration (geographic atrophy) 
over a 5-year period. The black outline shows the border of the atrophy and the presence of spared 
areas within it. (a) Baseline: there were two areas of geographic atrophy, partly surrounding the 
fovea. Visual acuity was 20/40. (b) One-year follow-up: the atrophy enlarged and coalesced into 
a horseshoe-shaped area of atrophy and corresponding dense scotoma. The fovea was spared, as 
was the area immediately above the fovea. Visual acuity was still 20/40, but the reading rate 
dropped by more than 50%. (c) Two-year follow-up: the atrophy has enlarged and the spared 
vertical region has gotten narrower. The fovea is still spared. Visual acuity was 20/60. (d) Three- 
year follow-up: the atrophy has enlarged and coalesced superiorly, so that there is a ring of atrophy, 
with two small spared areas within it. Visual acuity was still 20/60. (e) Five-year follow-up: the 
atrophy has enlarged further, and there is only a tiny slit of fovea spared within the atrophy. Visual 
acuity was 20/200. The patient now used eccentric retina for reading text that was 20/600 or 
greater in size 



first develop in areas near, but not involving, the fovea (Fig. 5.3a). As these atrophic 
areas enlarge and coalesce, the patient may develop a horseshoe of blind area around 
the fovea, and then a ring of scotoma surrounding the fovea (Fig. 5.3b) [45]. These 
patients may have good visual acuity, but they have difficulty reading and recognizing 
faces because the whole word or face does not "fit" in the spared seeing area that is 
surrounded by scotoma (Fig. 5.3c and 5.3d) [52]. Eventually, the fovea itself 
becomes atrophic and severe visual acuity loss occurs (Fig. 5.3e). Unlike wet AMD, 
there are no treatments available to slow or prevent vision loss in GA at the present 
time. Once an area becomes atrophic, the photoreceptor cells die, so that a potential 
treatment that prevents cell death will not restore vision to an atrophic area. 

Patients with GA and other forms of advanced AMD have profound reductions 
in vision in dim illumination due to reduced contrast sensitivity, and require 
increased lighting in order to read [53]. They are also more sensitive to the effects 
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of glare and bright sunlight, requiring solar shields with tinted lenses outdoors, and 
in some cases indoors as well. Unlike advanced RP patients, the vast majority of 
individuals with advanced AMD do not require use of a white cane or mobility training 
since the peripheral vision remains intact even in late stages of the disease. 

Patients with other forms of macular disease, such as Stargardt disease (the most 
common form of macular degeneration in young people) and diabetic retinopathy, 
have similar symptoms to those described above. Optimizing the visual performance 
of patients with macular disease generally involves improving the lighting, increasing 
contrast, and teaching the patient how to adapt to the presence of scotomas in the 
central visual field. 



5.4 Charles Bonnet Syndrome 
5.4.1 Overview 

Hallucinations are organized perceptions in the absence of an external stimulus. 
Charles Bonnet Syndrome (CBS) is a condition first described by Charles Bonnet 
in 1760, in which visual hallucinations occur in visually impaired individuals. The 
patient is aware that the hallucination is not reality and there is an absence of 
cognitive impairment. CBS is usually associated with impaired vision due to reti- 
nal degeneration, and can be present in either AMD or RP patients. It is possible 
for CBS to occur with any type of ocular pathology, but it is less common in 
patients with glaucoma, optic neuritis or cataracts than in those with retinal dis- 
ease. It is more common in the elderly. A diagnosis of CBS can be made when 
alternative conditions known to give rise to hallucinations such as migraine, 
occipital lobe epilepsy, and psychiatric disease have been excluded. If other senses 
are involved (e.g. hearing, smell), then it is unlikely to be CBS. 

The prevalence of CBS has been estimated up to 15% [44], but likely varies due to 
patients' concern to hide their symptoms for fear of being labeled psychiatrically 
unstable. CBS is very rare in those without vision loss, with only monocular loss or in 
those with no light perception in both eyes. The onset of some cases has been reported 
prior to vision loss. Some patients have found a reduction in the occurrence of CBS 
when their vision loss progressed significantly or led to complete blindness. Cases of 
CBS have also been reported following enucleation or removal of the eye [43]. 

The visual hallucinations can involve photopsias, colors, moving parts, patterns 
or shapes (simple) or well-defined forms, images and scenes (complex) [1]. They 
tend to appear centrally, within the areas of vision loss, and about half of AMD 
patients report that the visual hallucinations are clearer than their current vision. 
Examples of the most common types of visual hallucinations include images of 
people or faces, followed by geometric patterns, and plants, flowers or trees [1, 30]. 
They are normally not frightening. For example, one of our patients reported that 
she sees people in her apartment. She checks that her door is still double-locked, 
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goes over and says hello, and then gets back to what she was doing. Some actually 
give great pleasure. One woman with advanced macular degeneration reported she 
saw little girls with beautifully embroidered aprons. She enjoyed watching them, 
and knew that it must be a visual hallucination because if she were truly using her 
visually-impaired eyes, she would not be able to see the fine embroidery. One 
woman lost 25 pounds because she kept seeing bugs in her food. One man thought 
he was seeing saints, because he saw many different faces. However, the persistence 
or frequent recurrence of episodes of CBS can become annoying to patients. 



5.4.2 Complexity of Visual Hallucinations in CBS 

Using functional magnetic resonance imaging (fMRI) of the visual cortex, simple 
visual hallucinations have been found to originate early in the visual pathway (VI 
and/or V2), whereas more complex visual hallucinations were generated in the 
higher visual areas [9]. Only a very small proportion, 8%, of the reported visual 
hallucinations was restricted to the area of binocular field loss [ 1 ] . 

In patients with bilateral scotomas secondary to AMD, the likelihood or complexity 
of CBS is not predicted by the extent of visual acuity loss [ 1 ] . The rate of vision loss 
may be a predictor of CBS, as cases of AMD patients with rapid loss of vision due 
to significant exudation or laser photocoagulation have been reported. Cognitive 
factors, such as state of arousal, may play a central role in the development of CBS 
once the vision loss has reached a critical threshold level, either in terms of the extent 
of visual field loss or amount of reduction in visual cortical processing. 

The neural basis underlying CBS remains under debate, with many hypotheses 
proposed [37, 40]. Initially CBS was thought to be related to epileptic discharges, 
but imaging did not confirm this hypothesis. Another etiological theory is that it 
may be a deafferentation phenomenon, such as a visual analogue to a phantom limb 
in amputees, in which brain activity occurs without sensory input. The significant 
biochemical changes in the areas of the deafferented synapses may result in a 
release phenomenon or hyperexcitability response. 



5.4.3 Predictors and Alleviating Factors for CBS 

The mean age of onset of CBS ranges from 75 to 84 years [37], however, cases 
have also been reported in much younger individuals, including children. Fewer 
cases among children may reflect an increased plasticity of the immature afferent 
pathway and/or an inability of the patient to understand or describe the visual experi- 
ences. Younger age among AMD patients has been identified as a potential predictor 
for CBS [1]. CBS is more common among women; however this likely reflects 
the female bias of an elderly population, as well as a greater willingness among 
female patients to report visual phenomena that may be considered "abnormal" by 
caregivers. 
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CBS is more common in those who live alone and have limited social contacts. 
Other risk factors for CBS include loss of energy, low extroversion, shyness, use of 
beta-blocker medications, loneliness or bereavement [55]. Typically an episode 
of CBS is brief, lasting only a few seconds to minutes, and they tend to occur more 
often in the evening. In one study, about half of the patients reported that their 
visual hallucinations typically lasted between 1 and 60 min [44]. About a quarter 
to a third of patients with CBS experience visual hallucinations daily, whereas 
about half experience them weekly or monthly. The frequency and durations of the 
visual hallucinations can vary within and between individuals. They tend to last 
longer when the individual is drowsy or tired, indicating a relationship with the 
patient's state of arousal [54]. As with photopsias, some potential triggers for CBS 
include stress and fatigue. CBS is more likely to occur in low levels of illumination, 
whereas photopsias are associated with either absence of light or bright light, but 
more often with bright light [37]. 

Patients report that they have little control over the appearance or duration of the 
images. Some patients are able to stop their hallucinations through rapid closing 
and opening the eyes, blinking, sustained eye closure, turning on a light, looking at 
something else for distraction, walking away, or hitting or shouting at the hallucina- 
tion when alone [12]. Potential treatments may include optical devices, such as 
prisms or telescopes, tinted lenses, use of night lights in the bedroom or increasing 
social contacts [37]. There are some case reports indicating that some of the atypical 
antipsychotic or antiepileptic medications can alleviate symptoms; however their 
effectiveness in clinical trials has not been established [13, 44]. 



5.5 Filling-In Phenomena (Perceptual Completion) 

In order to explain the visual deficit in macular degeneration to people with normal 
vision, an image is often presented showing a black splotch in the middle of the 
picture. While this conveys the fact that it is the central vision that is involved, 
most patients do not see a black splotch. Instead, they report that things are blurry. 
Patients seem to be able to distinguish this type of blurriness from the difficulty of 
seeing small print, for example. The blurriness is a result of the patient's brain and 
visual system trying to "fill-in" what is not seen [41]. The area corresponding to 
the scotoma cannot look clear, because in truth the image is not being seen. But 
the image is completed, and the patients are often not aware that the reason they 
cannot recognize a face is that parts are really missing. 

The filling-in phenomenon or perceptual completion has been associated with 
the difficulty appreciating a scotoma on Amsler grid testing. The Amsler grid is a 
square of graph paper, subtending 20° horizontally and vertically at the defined 
viewing distance. There is a large dot in the middle of the grid. The way in which 
this test is generally used is to try to have the patient center the eye on the grid (for 
example, by seeing the four corners of the grid), and then report if the central dot 
can be seen. (Alternatively, the patient is told to look directly at the dot). The task 
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is to determine if any of the lines of the grid are distorted, wavy, or missing. The 
Amsler grid often is used to try to help patients with AMD detect the early develop- 
ment of wet AMD by detecting a distortion in the grid. However, patients who have 
definite blind spots on visual field testing often cannot detect any part of the grid to 
be missing or distorted. One study found that 40% of patients with absolute scotomas 
in their central field, from the heavy laser treatment used for macular degeneration 
in the past, could not detect a defect on Amsler grid testing [14]. This was further 
examined by placing the grid directly over the scotoma in a scanning laser ophthal- 
moscope, in which the examiner could see that it was directly over the blind area. 
The patients still could not detect the defect on the Amsler grid testing [46]. 

There are ways to make the patient aware of where filling-in is taking place. In 
a technique using face fields [49], the patient is instructed to cover one eye. The 
patient is told to look at the nose of the examiner, so that the nose is seen as clearly 
as possible. While the patient is looking at the nose, he/she is asked if there is any 
part of the face that is blurry, distorted, or missing. The face is a very salient stimulus, 
and patients are often able to say that an eye is missing, or a piece of the cheek is 
blurred, etc. This is very helpful for defining the location of the preferred retinal 
locus of fixation (see section 5.7), to allow for more effective low vision training. 



5.6 Remapping of Primary Visual Cortex in Patients 
with Central Scotomas from Macular Disease 

Much of the primary visual cortex is devoted to representing the macula [28, 48]. 
When there are central scotomas, so that much of the macula is blind, what happens 
to the corresponding areas of primary visual cortex? [25]. Do they remain silent, 
are they somehow recruited by surrounding retinal areas, or are they stimulated by 
other cerebral areas? Certainly, cortical areas corresponding to other sensory 
modalities do not remain silent. For example, there is the phantom limb phenomenon 
following amputation [17]. For vision, there is the additional question as to whether 
the potential remapping of the visual cortex could contribute to the use of an eccen- 
tric retinal locus of fixation as a "pseudofovea" [59]. 

Functional MRI (fMRI) can be used to perform retinotopic mapping of the 
visual cortex [12]. One can have the patient observe a given pattern on a monitor 
and measure which cortical areas are stimulated by the differential amount of oxy- 
gen consumption in each area. Interest has focused on whether there remain silent 
areas in the primary visual cortex, corresponding to the macular scotomas, or 
whether there is remapping and therefore activity in these areas that previously 
subserved the now scotomatous macular region. Work in this area has been ongoing 
for only the past 5 years. Current information suggests that cortical remapping 
occurs only when the fovea itself is scotomatous. When the fovea is seeing, the 
visual cortex corresponding to a scotoma in the macular region near it does not 
appear to have remapping [3, 4, 36, 51]. One paper reported more stimulation of 
the so-called lesion projection zone when the eccentric preferred retinal locus of 
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fixation was stimulated [47]. The stimulation of the lesion projection zone may be 
task-related [36], and may be from higher cortical levels. When patients with cen- 
tral scotomas were shown scrambled patterns, there were silent areas in the primary 
visual cortex. When the same patients were shown faces, the formerly silent areas 
were stimulated, and the face-specific extrastriate cortex was stimulated as well 
(Rosenau BJ, Greenberg AS, Sunness JS, Yantis S. Cortical Lesion Projection Zone 
activity in Retinal Disease Patients is Caused by Object-Specific Feedback, not 
Plasticity, presented at the 2008 VSS Meeting). The hypothesis is that there may be 
top-down stimulation of primary visual cortex. This may relate to the filling-in 
phenomenon described above. 



5.7 The Preferred Retinal Locus for Fixation 

When the fovea is no longer functional, the patient must use an eccentric retinal 
area to fixate and view the object of interest. Patients vary widely in their ability to 
create a "pseudofovea", that is an eccentric preferred retinal locus (PRL) to which 
the oculomotor system can direct the object of interest [59]. It is not understood 
why some patients are able to adopt an eccentric PRL effectively and read a chart 
by looking up or to a side, while other patients with similar scotomas must use 
repeated scanning movements, report letters coming in and out of view, and have 
much greater difficulty in reading. 

There is evidence that adoption of an eccentric PRL can improve visual acuity. 
In a prospective study of patients with bilateral advanced dry AMD (geographic 
atrophy) followed for 3 years, 17% improved their visual acuity by at least 
0.2 logMAR (i.e., seeing letters 2/3 the size or smaller) over the course of the 
follow-up [50]. This improvement occurred only in the worse-seeing eye at base- 
line, and occurred despite continual enlargement of the central scotoma over the 
3-year period. Scanning laser ophthalmoscope analysis showed that at baseline, 
these worse-seeing eyes could not use peripheral retina effectively; the fixation 
cross was placed within the scotomatous area where it was not seen, and it could 
not be stably placed on seeing eccentric retina. At 3 years, these worse-seeing 
eyes had acquired the ability to use an eccentric retinal locus for fixation, with 
consequent improvement in visual function. The same improvement did not happen 
in the better-seeing eyes, presumably because these eyes already were using 
eccentric retinal fixation loci at baseline. With both eyes open, the better-seeing 
eye's fixation pattern likely dominated the two eyes, perhaps interfering with the 
development of a monocular PRL in the worse-seeing eye. By 3 years, with worsening 
of the better eye as well, more attention was now directed to the worse-seeing 
eye, with improved use of peripheral seeing retina and consequent improvement 
in visual function. 

This spontaneous improvement in visual acuity in the worse-seeing eye has 
important implications for future clinical trials for retinal prostheses. In all 
likelihood, the eye to be treated initially for each patient will be the one with worse 
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visual acuity. The attention itself that is directed to this eye by virtue of the 
intervention may improve the patient's ability to use peripheral retina and thereby 
improve the visual acuity. This phenomenon may be addressed in part by providing 
some low vision training prior to the clinical trial. 



5.8 Photopsias 

5.8.1 Photopsias in RP 

The basis of photopsias, or light flashes, in RP and other conditions is not well- 
understood. They may be manifestations of spontaneous activity in compromised 
retinal cells, or in retinal microneuromas, triggered through inner plexiform layer 
connections, possibly due to remodeling and/or ganglion cell and axon loss in the 
degenerating retina. Photopsias may be linked in important ways to the processes 
occurring during retinal implant stimulation, and their characterization may be 
helpful for the future development of prosthetic vision. Photopsias may interfere 
with visual function testing during clinical trials, as well as RP patients' vision 
while performing daily activities, underscoring the importance of their character- 
ization among this patient population. 

A survey of RP patients conducted in the clinic indicated that 35% reported 
flashes of light [24]. A more recent internet-based anonymous survey of pho- 
topsias in RP patients found that 93% of those who completed the survey had 
experienced photopsias. The photopsias in this survey were described as phos- 
phenes (slow, localized dots or shapes) by 71%, flashes (all or most of the field at 
once) by 58%, static noise (like on a television without reception) by 31%, and 
fluorescence (a background glow) by 20% of those who noted photopsias [6]. 
Photopsias were most commonly reported to have a shape of a crescent, arc or 
semi-circle by over half of the respondents. The factors that were most commonly 
reported to be associated with an increase in photopsias were bright light, fatigue, 
stress, exercise and absence of light. 

Photopsias are commonly noted by RP patients in both the earlier stages of the 
disease, as well as in those with end-stage retinal degeneration. Nearly half of those 
who have photopsias experienced them before they were diagnosed with RP, and 60% 
stated that they first noticed photopsias when they were less than 30 years of age [6]. 

RP patients who were able to read normal or small sized font without magnifica- 
tion, were driving currently, or who could easily navigate or had only some diffi- 
culty with mobility in unfamiliar areas, were two to three times more likely to note 
photopsias mostly or only peripherally versus in their central vision. Therefore, the 
extent and location of photopsias appear to be related to residual photoreceptor 
function assessed by self-reported vision and performance of daily living activities. 
Photopsias tend to start in the periphery early in RP and then later occur more 
centrally and in areas with vision as deficits in visual function occur. Therefore RP 
patients may become more aware of photopsias as vision loss becomes more 
advanced. 
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The majority of RP patients indicated that photopsias interfere with their vision, 
and interference was more likely when photopsias occurred daily, increased in 
frequency over time, or were located across a larger area over time. About half of 
RP patients who report photopsias experience them daily. The increased frequency 
of photopsias in RP appears to be related to increased perceived stress and decreased 
positive mood. About a quarter experienced photopsias constantly, and over half 
experienced photopsias for only a few seconds at a time. The location or frequency 
of photopsias in later RP stages may obstruct vision at times, and is a potential issue 
for patients' function or when obtaining vision measures. 



5.8.2 Photopsias in AMD and Other Ocular Diseases 

One report indicated that photopsias are common in patients with macular choroidal 
neovascularization [8], occurring in 59%, and of those, 59% experienced white 
colored photopsias and described them as typically lasting several seconds. 
Subretinal fluid, cicatrix formation and larger disciform scars were more common 
among individuals who noted photopsias than in those who did not. The occurrence 
of photopsias among those with macular choroidal neovascularization may poten- 
tially be due to sensory deprivation; when normal input to the visual system is 
repressed and the activity of other neural tissue may become more apparent. 

Rare retinal diseases with scotomata and the possible presence of photopsias 
are acute zonal occult outer retinopathy (AZOOR) [61], multiple evanescent 
white dot syndrome (MEWDS) [29], acute idiopathic blind spot enlargement 
(AIBSE) affecting the retina around the optic nerve without optic nerve head swelling 
or choroiditis [16], autoimmune retinopathy (including cancer- and melanoma- 
associated retinopathies) [23], photoreceptor dysfunction due to digitalis toxicity 
[38], and punctate inner choroidopathy (PIC) [19]. Some patients with optic neuritis 
secondary to multiple sclerosis [11, 33] and restrictive thyroid ophthalmopathy 
with tight inferior recti eye muscles [10] have also reported photopsias associated 
with eye movements, likely related to compression or traction. 



5.9 Concluding Remarks 

In addition to the anatomical and functional changes that occur in the retinal and 
visual cortex, it is important to consider and address the various types of changes 
in vision experienced by patients with RP and AMD that will impact both the 
objective and subjective outcomes with prostheses. In particular, emphasis should 
be placed on the patients' perspectives of functioning with a retinal degenerative 
disease, considering the disability and uncertainty associated with the performance 
of activities of daily living and day-to-day visual fluctuations. In the assessment 
and rehabilitation of prosthetic vision, several types of visual phenomena that may 
ordinarily occur in those with advanced AMD or end-stage RP will potentially 
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interfere with the visual percepts produced by prosthetic devices. These aspects 
need to be better understood and managed by researchers and clinicians working in 
the area of prosthetic vision. 
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Part II 
Neural Stimulation of the Visual System 



Chapter 6 

Structures, Materials, and Processes 

at the Electrode-to-Tissue Interface 

Aditi Ray and James D. Weiland 



Abstract This chapter reviews the basic concepts of neural stimulation along with 
safety considerations for both the electrode and tissue. The section on electrode- 
electrolyte interface describes the basic mechanism of charge injection at the inter- 
face introducing the reader to the electrode double layer. The use of circuit models 
to represent the physical processes at the interface and in the bulk tissue is discussed. 
The next section provides a detailed description of the biopotential electrode along 
with measurement techniques used in electrode characterization. Following this, an 
overview of popular electrode materials for neural stimulation is provided for the 
reader. These include conventional materials such as platinum and iridium oxide, 
as well as newer materials like conducting polymers and carbon nanotubes. The 
next section reviews the concept of extracellular stimulation introducing the reader 
to Goldman Equation used to describe the membrane potential. Finally the section 
dedicated to safe stimulation of tissue describes the mechanisms of neural injury 
and parameters considered to ensure safe neural stimulation. Special emphasis is 
placed on safety studies of retinal stimulation. 
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IHP Inner helmholtz plane 

OHP Outer helmholtz plane 

PBS Phosphate buffered saline 

PEDOT Poly(3,4-ethylenedioxythiophene) 

PSTHs Post stimulus time histograms 

SIDNE Stimulation induced depression in neuronal excitability 

SIROF Sputtered iridium oxide film 

TiN Titanium nitride 

TIROF Thermal iridium oxide film 



6.1 Introduction 

Electrodes used for neural stimulation must operate under demanding conditions. 
Along with exhibiting biocompatibility, they have to be small enough to cause 
localized excitation of the target neurons and large enough to support safe delivery 
of charge for effective neuronal excitation. Also in most cases, the implanted elec- 
trodes are required to function for the lifetime of the implant recipient. Consequently, 
for any neuroprosthesis employing electrical stimulation to be successful, the 
implanted electrodes must function for decades without significant degradation or 
damage to either themselves or to the tissue. This warrants understanding the char- 
acteristics of the metal-tissue interface in an effort to optimize electrode material 
selection and design stimulation protocols. Seminal studies of the interface were 
performed as part of larger consortia developing neural prostheses for paralysis and 
for implantation in the visual cortex. While retinal prosthesis development has 
benefited from these findings, the unique structure of the retina and eye require 
special consideration. Hence, increasing efforts are being made to understand the 
safety requirements of such retinal prostheses. 

Material presented here has been mainly derived from three sources: Principles 
of Neural Science by Kandel, Schwartz and Jessell [30], Electrochemical Methods: 
Fundamental and Applications by Bard and Faulkner [3] and Electrical stimulation 
of excitable tissue: design of efficacious and safe protocols by Merrill [39]. 



6.2 Electrode-Electrolyte Interface 

Whenever a metal electrode is placed in an electrolyte, thermodynamic processes 
operate to bring the two phases in electrochemical equilibrium. This causes attrac- 
tion between the charge carriers in the two phases leading to the formation of a net 
potential across the interface. This interface is popularly known as the electrical 
double layer with the principal charge carriers in the metal phase being the elec- 
trons and those in the electrolyte being the ions. The importance of this interface 
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lies in the fact that for any neural excitation to take place, current has to flow 
through tissue. Hence, the key to understanding and controlling stimulation through 
metal electrodes lies in understanding the different electrochemical processes that 
take place at the electrode-electrolyte interface. 

When a metal electrode is placed in an electrolyte, a finite separation of charge 
occurs leading to the formation of the electrical double layer. This charge separation 
has several manifestations. One reason for charge redistribution at the interface is ions 
in the electrolyte combining with the electrode. This leads to a net transfer of electrons 
between the two phases causing a plane of charge at the metal electrode that is opposed 
by a plane of charge in the electrolyte. Other reasons for the formation of the double 
layer include the specific adsorption of certain chemical species and preferential ori- 
entation of polar molecules such as water. The solution side of the double layer is 
composed of several layers. The inner layer called the Helmholtz or Stern layer con- 
sists of solvent molecules and some other species such as specifically adsorbed ions or 
molecules. The locus of electrical centres of the specifically adsorbed ions defines the 
inner Helmholtz plane (IHP) while the locus of centres of the nearest solvated ions 
defines the outer Helmholtz plane (OHP). The solvated ions are said to be non-specifically 
adsorbed as their interaction with the charged metal is independent of the chemical 
properties of the ions. These ions are distributed in the three dimensional region called 
the diffuse layer extending from the OHP into the bulk solution. The thickness of the 
diffuse layer is dependent upon the total ionic concentration of the solution. 

The metal electrode- solution interface has been shown to behave like a capacitor 
with a finite amount of charge residing in a very thin layer on the metal surface (excess 
or deficiency of electrons). In the solution side, the charge is made up of excess anions 
or cations residing close to the electrode surface. At any given potential, the double 
layer is characterized by its double layer capacitance C dl ( 10-40 u.C/cm 2 ). 



6.2.1 Basic Mechanisms of Charge-Injection Across 
the Electrode-Electrolyte Interface 

Before proceeding into understanding the basics of neural stimulation and elec- 
trode characterization, it is worth noting the different terminologies assigned to 
the electrodes employed, which vary depending upon the experimental condi- 
tions. For electrochemical characterization, a three-electrode system is employed 
where the electrode of interest is referred to as the working electrode, while the 
other two are called the counter and reference electrodes. For neural stimulation, 
a two-electrode system is employed where current enters the tissue through the 
stimulating electrode and exits the tissue through the return electrode. Neural 
stimulation can be further subdivided into monopolar and bipolar stimulation. 
Monopolar stimulation uses a small stimulating electrode and a large return 
electrode while in bipolar stimulation, two small electrodes are used as the source 
and sink in an effort to focus the current to small regions such as nerve cuff elec- 
trodes and electrodes used in cochlear implants. Measurements may contain a third 
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electrode termed the reference electrode required for measuring precisely 
controlled electrical potentials. At equilibrium (no current), the potential of the 
system remains constant and is typically referred to as the open-circuit potential. 
Net electrochemical processes begin to take place as soon as the potential is 
forced away from equilibrium and resulting current begins to flow through the 
system. Charge transfer across the interface takes place through two primary 
mechanisms viz., Faradaic and non-Faradaic reactions. 

Faradaic and non-faradaic reactions: Non-Faradaic processes include redistribu- 
tion of the charge at the electrode-electrolyte interface and do not involve any net 
transfer of charge species across the interface. If charge injection is achieved 
through only non-Faradaic reactions, i.e. charging and discharging the double-layer 
capacitance, then the electrode-electrolyte interface can be modeled as a simple 
capacitor, viz. the double-layer capacitor C„. If the total amount of charge trans- 
ferred is small then the transferred charge can be recovered by simply reversing the 
polarity of the applied pulse or by discharging the capacitor. In addition to charging- 
discharging of the double layer capacitance, charge injection can also be achieved 
by Faradaic processes such as oxidation-reduction reactions. These reactions 
involve the transfer of electrons between the two phases of the reaction and unlike 
the capacitive mechanism, may or may not be completely reversible in nature. 
In case of reactions in which at least one of the chemical species is surface bound, 
the reaction is completely reversible under steady state conditions. Such reactions 
are limited by the available surface area of electrode and the amount of species 
adsorbed onto the interface. However, reactions that do not involve at least one 
surface bound species, have no mechanism to force the reaction to be reversible in 
the steady state. Charge balancing in the Faradaic regimen is most often achieved 
via multiple partially reversible reactions that result in the release of one or more 
possibly cytotoxic chemical substances in the surrounding tissue. 



6.3 Electrode Material 

In order to depolarize neurons or to record biological potentials, an interface is 
required between the body and the electronic apparatus. This interface is called the 
biopotential electrode. Biopotential electrodes deal with challenges different from 
electrodes used in other systems. First the electrode material has to be biocompatible, 
i.e. non-toxic to the body and second it has to have the ability to serve as a trans- 
ducer. This is because as we saw in the preceding section, current in the electrode 
is carried by electrons while in the electrolyte it is carried by ions. 

Electrode potential: When a metal is brought into contact with a solution, a net 
rearrangement of charge occurs at the interface leading to a loss of neutrality of 
charge at the interface. As a result, the electrolyte in the immediate vicinity of the 
electrode is at a potential different from the rest of the solution. This difference in 
potential is called the half-cell potential and is determined by many different param- 
eters such as the type of metal, the type and concentration of ions in the solution, 
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temperature, etc. This half-cell potential is also referred to as the electrode interfacial 
potential. It is not possible to measure this potential without utilizing a second 
electrode. However, the second electrode would then create an interface of its own 
with the electrolyte thus making it impossible to separate the two resulting poten- 
tials from each other. To overcome this, electrochemical cells are evaluated in their 
entirety, generally composed of a working electrode and a reference electrode sepa- 
rated by the electrolyte. Thus a cell's potential is defined as the potential of the 
working electrode vs. the reference electrode. 

Consider the reaction between a metal electrode and a redox couple in the 
electrolyte: 

0+H<T <r^> R 

The equilibrium potential for any electrochemical cell can be calculated using 
the Nernst equation: 

E t =— lni-f (6.1) 

nF [X], 

where, [X] and [X]. are the concentrations of the species, R is the gas constant, 
T is the absolute temperature (Kelvin), F is Faraday's constant and n is the number 
of electrons transferred. For the electrochemical cell above, if the concentration of 
both species in solution is equal then the potential of the cell will equilibrate to its 
formal potential EP. For unequal concentrations, using the Nernst equation, the 
equilibrium potential for the electrochemical cell is: 

E =E o + ^L ln M (6.2) 

nF [R] 

In the absence of any net current, the measured cell potential is called the open- 
circuit potential, which again is the sum of the two interfacial potentials. Now if 
instead a current is present, then the observed potential is different from the equi- 
librium potential. This is due to the polarization of the electrode and the difference 
between the observed potential and the equilibrium potential is known as the over- 
potential r\. 

n=E-E eq (6.3) 

Three basic mechanisms contribute to overpotential: ohmic, concentration and 
activation overpotentials. Ohmic overpotential is due to the electrolyte resistance 
which leads to a voltage drop across the solution during the passage of current 
between the electrodes. Concentration overpotential occurs due to changes in the 
distribution of ions at the electrode-electrolyte interface. Activation overpotential 
occurs due to charge transfer processes involved during oxidation-reduction reac- 
tions that are not completely reversible. The net overpotential is simply a sum of 
three mechanisms. 
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Polarizable and non-polarizable electrodes: For ideally polarizable electrodes, no 
actual charge crosses the electrode-electrolyte interface during current flow. 
Instead, during current flow, redistribution of ions occurs at the interface thus 
exhibiting capacitor like properties. As a result the overpotential is dominated by 
the concentration overpotential. One example is titanium nitride electrode where 
charge injection takes place through capacitive charging-discharging processes. 
Noble metals such as platinum also behave as polarizable electrodes but over a 
limited range of voltages. Ideally non-polarizable electrodes on the other hand are 
the ones in which current passes freely between the electrode-electrolyte interface 
and hence causes no overpotential. Electrodes such as silver-silver chloride and 
saturated calomel come closest to behaving as non-polarizable electrodes. These 
electrodes are best used as reference electrodes during measurement of electrode 
potential as there is no change in voltage across their interface during current flow. 
However, it is essential to note that in reality no electrode behaves either as ideally 
polarizable or ideally non-polarizable. Electrodes come closest to ideal characteris- 
tics only over a limited range of voltages. 



6.3.1 Electrode Characterization 

Measurement of impedance: Electrochemical impedance spectroscopy [17] has 
been used successfully to characterize the electrode-electrolyte interface. Specifically 
for neuroprostheses employing current stimulation, impedance measurement tech- 
niques have been employed to test the efficacy of neural stimulation. Studies in the 
past have shown that for all stimulation strategies to efficiently inject charge across 
the electrode-tissue interface, an optimal relationship exists between the threshold of 
excitation and the distance between the electrode and tissue. For the auditory brain- 
stem implants [31], measurements of threshold of excitation as a function of the 
distance of the electrodes from the target neurons have shown a strong correlation 
between the two [42] . The reason behind this is that in order to cause neuronal exci- 
tation a minimum amount of current density is required. If the interface impedance 
were high, it would lead to a higher applied voltage, which could then become a 
limiting factor in the power capabilities of the device. As shall be discussed in 
Sect. 6.5, in some cases, this high voltage can also lead to undesirable electrochemical 
reactions to take place at the interface thereby causing tissue damage. 

As it is not possible to control the tissue properties of the target system, efforts 
are made instead to control the electrode design in order to allow safe and effective 
stimulation. To achieve this, equivalent circuit models of the electrode-electrolyte 
interface have been developed which along with impedance measurements, provide an 
estimate of optimal parameters for the electrode. The first ever model was proposed 
by Warburg in 1899 who modeled the interface as a polarization resistance in series 
with a polarization capacitance. This would produce a straight vertical line on the 
complex plane plots (Z imaginary vs. Z real). However, for solid electrodes it was 
often observed that the straight vertical line had an angle less than 90°. Thus, the 
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electrode impedance consisted of a polarization resistance in series with complex 
impedance exhibiting frequency dependency. The phenomenon of constant phase 
angle was first shown by Fricke and the impedance associated with it is termed as 
the constant phase element (CPE). CPE is thought to arise from surface inhomoge- 
neities and slow reaction kinetics [5]. Mathematically, CPE is represented as: 



1 



?W 



(6.4) 



where T is a constant in Fcm" 2 s _1 and <p is related to the angle of rotation of a purely 
capacitive line on the complex plane plots. The CPE is often used to represent a 
"leaky capacitor" and only when <p = 1, T = C d| and a purely capacitive behavior is 
obtained [33]. Equation (6.4) can be used to model the Warburg element that 
accounts for diffusion delay in Faradaic currents by assigning (j) = 0.5. Finally (6.4) 
can be used to describe a pure resistor for (j) = and a pure inductor for = -1. 
Randles's work showed the importance of the impedance associated with the fara- 
daic processes occurring at the electrode-electrolyte interface. The popular Randies 
model consists of an interface capacitance shunted by charge transfer resistance 
(R CTc ) in series with the solution resistance (R„ ) (Fig. 6.1) [21]. Since then studies 
have been done to characterize different electrode materials and their surfaces 
based on different combinations of the Randies model, constant phase element and 
Warburg impedance [19, 25, 54]. As platinum is the most widely used electrode 
material for biomedical applications, groups have focused on extensively character- 
izing its properties. Frank et al. used EIS techniques to compare three electrode 
materials geared towards biomedical applications: platinum, platinum black and 
titanium nitride [19]. 

The electrochemical impedance theory describes the response of a system to an 
alternating current or voltage input as a function of frequency. The basic approach 
of EIS is to apply small amplitude perturbations (sinusoidal current or voltage sig- 
nals) to the electrodes and measure the system's current or voltage response. For 
microelectrodes used for neural stimulation, usually a sinusoidal voltage signal is 
used as the excitation signal and the resulting current is measured as the response 
of the system (potentiostatic EIS). Typically the single-sine technique is used where 
in the excitation signal is applied at discrete frequencies and the resulting response 
signal is measured at each frequency to develop the impedance spectrum. In most 
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Fig. 6.1 Equivalent circuit model of electrode-electrolyte interface. R solution resistance; 
R CT charge-transfer resistance; Z w Warburg element; Z cpE constant-phase element 
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experiments, measurement is started at the highest frequency and stepped down to 
progressively lower values until enough data has been collected to determine the 
impedance of the system as a function of frequency. This is done to ensure minimal 
sample perturbation, and to explore non-Faradaic before Faradaic charge transfer. 
The impedance profiles of microelectrodes assist in developing electronic models 
analogous to the electrode-electrolyte interface such as the Randies model 
described in the preceding paragraphs. Values of the different circuit elements can 
be estimated from the impedance measurements. This aids in designing improved 
versions of the electrode in order to achieve optimal charge-injection situations. 
The profiles are viewed either through the 'Nyquist plot' or the 'Bode plot' and 
corresponding model parameters can be estimated. At high frequencies, the imped- 
ance of the Randies cell becomes almost entirely dominated by the solution resis- 
tance R while at low frequencies the resistance of the electrochemical reaction R 

s ^ e 

also comes into play. The solution resistance has long been shown to have an 
inverse relationship with the radius of a disc electrode. However, recent work by 
Ahuja et al. suggests that this dependence may not hold for all frequencies [1]. 
Their work showed that the electrode impedance does scale with radius but only in 
the high frequency regime (-100 kHz), whereas at lower frequencies (~10Hz) it 
scales with the area of the electrode. Thus, only the electrode edge contributes at 
higher frequencies due to the primary current distribution while at lower frequen- 
cies, a secondary current distribution comes into play that drives the current to the 
centre of the disk leading to an area dependence. They also showed that for micro- 
electrodes of radii less than 50 urn, the area dependence is exhibited even at rela- 
tively higher frequencies due to the decreased RC time constant and double layer 
charging of the electrodes at these frequencies. 

Surface reactions and potential limits: Cyclic voltammetry (CV) falls under the 
class of voltammetric methods where the electrode potential is controlled and the 
resulting current is measured. In voltammetric methods, solutes in contact with 
the electrode undergo oxidation or reduction reactions producing current at the 
electrode surface that is measured. In case of cyclic voltammetry, the applied poten- 
tial is linearly varied with time (cycled) while the resulting current is measured. The 
applied potential has a triangular waveform with negative and positive turn-around 
potentials. Since, in a cyclic voltammogram, the range of applied potential is quite 
large, the measured current aids in understanding the reaction mechanisms available 
during stimulation. CV can characterize the potential at which the reaction proceeds 
maximally, the reaction kinetics, and the reversibility of the reaction, all of which 
are critical to determining if this reaction can be safely used to transfer charge to 
tissue. Platinum by far has been the most well studied electrode material (Fig. 6.2) 
but with increasing demands of neural stimulation treatment strategies, focus has 
shifted towards analyzing and characterizing other candidate electrode materials as 
reflected in the next section. 

For microelectrode characterization in neural stimulation applications, CV plots 
are used to study a number of important parameters associated with the safe and 
effective charge-injection at the electrode-tissue interface. 
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Fig. 6.2 Cyclic voltammogram of poly crystalline platinum in 1 M KOH at scan rate of 100 mV/s 
exhibits all the different processes involved during the cathodic and anodic direction. The potential 
scale is referred to a reversible hydrogen electrode (RHE) in the same solution. Reprinted from 
[28], with permission 



1 . Voltage limits. All electrodes must operate within the "water window," the term 
given for the potential range between hydrogen evolution potential (negative) 
and oxygen evolution potential (positive). 



2H 2 + 2e~ -+H 2 t +20H~ 



(6.5) 



2H 2 -> 7 T + 4/T + 4e~ 



(6.6) 



Cyclic voltammetry is used to determine these voltage limits, which are material 
and solution dependent. As will be discussed in Sect. 6.5, during neural stimula- 
tion only reversible reactions are employed for charge injection, to avoid causing 
damage to either the electrode or tissue. For example, from cyclic voltammo- 
grams of IrOx and TiN done by Weiland et al., it is observed that the water win- 
dow of TiN is in the voltage range of -0.6 to 0.8 V in phosphate-buffered saline 
solution (PBS). For IrOx, the water window has been estimated to be -0.7 to 
0.8 V in PBS [55]. Note that the water window does not change in width, but can 
shift depending on the reference potential. 
2. Charge-Injection mechanism. In a CV plot, the presence of peaks indicates 
electrochemical reactions occurring at the electrode-electrolyte interface along 
with the charging-discharging of the double layer capacitance. As an example, 
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the CV plot of platinum exhibits distinct peaks associated with the different 
surface reactions such as hydrogen-atom plating. Also, from the voltammograms 
of Weiland et al., IrOx CV traces exhibited distinct peaks indicating reduction- 
oxidation reactions involving transfer of electrons across the interface, along 
with current flow due to capacitive charging-discharging. On the other hand, the 
CV traces of TiN show no distinct peaks indicating that the current flow is domi- 
nated by the capacitive charging-discharging mechanism [55]. Also, from the 
nature of peaks, the type of reaction that is occurring can be determined. For 
example along with the oxidation-reduction peaks of water with the electrode 
metal, the presence of additional peaks indicates existence of other electro-active 
substances. 

3. Charge-storage capacity. An important parameter for neural stimulation is the 
charge-storage capacity of the electrode. This is determined by integrating the 
area under either the cathodic or anodic sweep in the CV plot within the water 
window. The value obtained indicates the maximum charge that can be injected 
via reversible surface processes by an electrode. This is usually expressed in 
terms of charge density limit of the electrode. As an example, the charge storage 
capacity of activated iridium oxide has been reported to range from 10 to 240 mC/ 
cm 2 depending upon the thickness of the film [52]. However, it should be noted 
that this is only capacity measured with cyclic voltammetry. The actual amount 
of charge injection that can be achieved during neural stimulation is usually only 
a fraction of the charge storage capacity and depends upon factors such as the 
thickness and morphology of the film, specific reactions of the redox material, 
pulse duration, etc. 

4. Reversibility of reaction. Whether the electrochemical reaction occurring is 
reversible or irreversible in nature can be determined from the cyclic voltammo- 
gram of the electrode. All chemical reactions, including reactions occurring at 
the electrode-electrolyte interface, proceed at a finite rate. The reversibility of a 
reaction is thus governed by the rate of electron transfer and surface concentra- 
tions. In a reversible reaction, the cathodic peak height is equal to the anodic 
peak height and the reversible half-wave potential will lie exactly midway 
between the peaks. However, as the reaction becomes more and more irrevers- 
ible, the cathodic peak height no longer remains equal to the anodic peak height 
and the separation between the peaks increase (Fig. 6.3). This situation can occur 
at high scan rates where due to slow reaction kinetics, the voltammogram changes 
from reversible to irreversible shape. 

Voltage response: While charge storage estimates acquired from CVs give the 
maximum charge value that the electrode in question can store without causing 
hydrolysis, the actual amount that is injected during current stimulation is quite 
different. Hence, in order to get a comprehensive picture of how the electrode will 
behave during active stimulation, one must study the voltage response developed 
during stimulation. Whenever a current pulse is applied across the electrode- 
electrolyte or electrode-tissue interface, a resulting voltage response develops 
across the interface. This voltage waveform is characterized by an initial drop 
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Fig. 6.3 Transition of cyclic voltammograms from reversible to irreversible domain. Case 1: 
reversible reaction with equal cathodic and anodic peak heights. Case 2: transition from reversible 
to irreversible reaction with increasing separation between cathodic and anodic peaks. Case 3: 
irreversible reactions indicated by large separation between the cathodic and anodic peaks. 
Modified from [18], with permission 



called the iR drop or the access voltage, which results from the ohmic losses in 
the system due to resistance of the electrolyte or tissue. These iR losses do not con- 
tribute to the potential difference across the interface that drives the charge across 
the interface. Hence, before performing analysis of the potential transients, it is 
essential to subtract these losses from the total voltage response. Based on the 
net potential across the electrode for a given pulse amplitude and duration, esti- 
mates of the actual charge injection capacities can be made. Also, by monitoring 
the voltage drop across the electrode, the safe charge injection limits can be estimated 
for voltage drops that do not exceed the water window of the electrode. 



6.4 Overview of Electrode Materials for Neural Stimulation 



An ideal candidate for electrode material for neural stimulation is one which is 
biocompatible, mechanically stable to surgical implantation, maintains its electrical 
and mechanical properties for the entire duration of use, is able to support the 
charge-injection requirements without inducing damage to itself or to the target 
tissue. Parameters that govern the efficacy and safety of electrode materials have 
already been described in the preceding section while parameters that govern the 
safety of biological tissue will be described in Sect. 6.5. In this section, brief over- 
view of materials that are most commonly used as electrodes for neural stimulation 
will be provided. 
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Platinum and its alloys with iridium is the most widely used electrode material 
for neural stimulation. Being a noble metal, it is highly resistant to corrosion and 
hence suitable for chronic implantations. The electrochemistry of platinum has 
been well studied along with its charge storage and injection capacities. Along with 
double layer charging, charge injection can occur through the reversible adsorption 
of hydrogen onto the platinum surface (H-atom plating) responsible for the pseudo- 
capacity of platinum. Brummer and Turner studied the underlying mechanisms 
during charge injection through platinum electrodes and its alloys [6-8] and found 
that these chemically reversible processes can provide charge injection up to 
300-350 |iC/cm 2 in simulated cerebrospinal fluid [8]. In practice, the safe charge 
injection limit of platinum depends upon a variety of factors such as the pulse dura- 
tion, current density and geometry of the electrode surface. For square pulses 
0.2 ms in duration, the safe charge injection limit with platinum was found to range 
from 50 to 150 |iC/cm 2 [48]. Some studies have attempted to increase the electro- 
chemical safe charge injection limit of platinum by increasing the real surface area 
of the electrode by roughening and have shown varying degrees of success [26, 57]. 

Iridium oxide belongs to the category of electrodes that are termed as valence 
change oxides. The oxide layer can be formed in three different ways. Anodic 
iridium oxide films (AIROF) are produced through repetitive potential cycling of 
the bulk metal between 0.0 and 1.5 V vs. a reversible hydrogen electrode in an 
acid or buffered neutral electrolyte [47]. The activated iridium is highly resistant to 
dissolution and corrosion and exhibit charge storage capacities ranging from 10 to 
240mC/cm 2 [52]. This charge storage capacity depends upon the thickness of the 
film and even moderate activation can lead to high values. However, during neural 
stimulation, only a fraction of this charge can actually be used. Weiland et al. found 
the reversible charge injection limits of AIROF to be about 4mC7cm 2 , which is 
greater than platinum and some other metals used for neural stimulation [55]. 
Beebe et al. showed charge injection limits of about 2mC/cm 2 for biphasic pulses 
and 3.5mC/cm 2 for monophasic pulses, 0.2ms in duration with activated iridium 
wire electrodes [4]. Iridium oxide films can also be formed by thermal decomposi- 
tion of layers of iridium salts (TIROF) or by reactively sputtering the oxide films 
onto a substrate from an iridium target (SIROF). Iridium oxide films on the whole 
have exhibited poor stability during chronic stimulation regimes however, recent 
work on SIROF shows improvement in in vitro stability during long-term pulsing [9]. 
In a separate study, Weiland et al. found the metal-tissue interface to be altered after 
chronic stimulation using thin film iridium oxide electrodes implanted in guinea pig 
cortex. They observed that current pulsing within safe limits increased the imped- 
ance at low frequencies (<100 Hz) after 1 or 2 days of stimulation and found the 
impedance change to correspond to a reduction in the charge storage capacity [54]. 
Other studies have also found iridium oxide electrodes to delaminate under high 
current pulsing with deposits in the surrounding tissue [10]. 

Capacitive electrodes are ideal for neural stimulation as they do not involve any 
reactions for charge injection and hence do not have to deal with problem of irre- 
versible reactions. However, these electrodes still have to be operated within the 
water window in order to avoid hydrolysis. The metal is insulated from the solution 
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by a thin layer of dielectric material that must be able to withstand the electric fields 
without any significant dc leakage. Materials in this group that have been found to 
be safe are anodized tantalum (Ta/Ta,0 5 ), anodized titanium (Ti/TiO,), thin films of 
barium titanate (BaTi0 3 ) and sputtered deposited titanium nitride (TiN). While 
anodized tantalum was found to have higher charge storage capacities than anod- 
ized titanium and thin films of barium titanate, titanium nitride was found to have 
charge storage capacities of 23 mC/cm 2 when combined with CMOS technology to 
develop microcolumnar structures [52]. However, the injectable charge limit of 
titanium nitride was found to be about 0.87mC/cm 2 for microelectrodes while for 
Ta/Ta 2 5 to be around 0.1-0.2mC/cm 2 for large electrodes [47]. Hence, capacitive 
electrodes though safer than electrodes employing Faradaic reactions, have in general 
lower charge injection capabilities when operating within the water window. 

Carbon nanotubes are also part of the capacitive electrode category exhibiting inter- 
esting electrochemical and mechanical properties. They are about five times stronger 
than steel and yet can be bent and twisted without breaking them. Recent work has 
shown them as potential electrode material for neural stimulation. Wang et al. devel- 
oped vertically aligned multiwalled carbon nanotubes (CNTs) using catalytic ther- 
mal vapour deposition system [53]. They tested the properties of the CNTs and 
found that CNTs have a higher charge injection limit of l-1.6mC/cm 2 after some 
surface treatment had been performed. Also, continuous pulsing did not degrade the 
properties of the CNTs. They also found these carbon nanotubes to be capable of 
causing neuronal excitation in embryonic rat hippocampal neurons. With its precise 
control of size, geometry and location by lithographic patterning of the catalyst and 
high charge injection capabilities without any Faradaic reactions, carbon nanotubes 
may be an answer to the requirements of neuroprostheses employing localized 
chronic neural stimulation. However, CNTs generally are formed at very high tem- 
peratures, making them incompatible with most batch electrode processes. 

Conductive polymers are one of the more recent members to the family of electrode 
materials for neural stimulation applications. Quite a few recent studies illustrate the 
feasibility of electrochemically polymerizing polypyrrole, polythiophene and their 
derivatives from aqueous solutions and depositing them on microelectrodes [12-15, 
31, 45, 46, 58]. Some of these studies have also shown that these polymers can suc- 
cessfully be incorporated with cell adhesion molecules, growth factors, etc. to fur- 
ther enhance their properties. With its superior electrochemical stability and 
biocompatibility, poly (3,4-ethylenedioxythiophene) or more commonly known as 
PEDOT may be well suited for chronic neural interfaces. Recent work suggests that 
PEDOT coatings can be deposited over platinum electrodes and be used for chronic 
neural stimulation [16]. The impedance of PEDOT coated electrodes was found to 
be lower than the bare platinum electrodes with corresponding lower voltage excur- 
sion to applied current pulses in PBS. However, the stability of the PEDOT coated 
electrodes under chronic stimulation regimes was found to depend largely upon the 
thickness of the coating that can be controlled through deposition time. Physical 
degradation and changes in microstructure of the film have been suggested as 
possible modes of failure. Hence, more work needs to be done to make these poly- 
mers successful electrode materials for chronic neural stimulation. 
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6.5 Overview of Extracellular Stimulation 

The bilipid layer membrane separates the intracellular region of the cell from the 
extracellular environment and acts as a barrier to the movement of ions between 
these two regions. It plays a crucial role in determining which ions are allowed 
to pass through and hence has the important properties of specificity and 
selectivity. The membrane also includes two specialized regions, the afferent 
region at which the neuron receives the signal and the efferent region at which 
the neuron sends the signal. 

All cells have a resting transmembrane potential (from hereon referred to as 
membrane potential) with the interior of the cell negative with respect to the 
exterior of the cell. This membrane potential is dependent on the concentration of 
the ionic species such that the equilibrium potential of each ion differs from the 
membrane potential. In general, the ions of interest are K + (potassium), Na + 
(sodium) and CI" (chloride). At rest, concentration of K + ions is higher inside giving 
it a negative equilibrium potential compared to the membrane potential. This gradient 
tends to move the ions out of the cell. The concentration of Na + ions on the other 
hand is higher outside than inside the cell giving it a positive equilibrium potential, 
which causes them to move into the cell. At rest, the membrane acts as a barrier and 
is less permeable to Na + ions compared to K + ions. The concentration ratios of these 
ions are maintained by ionic pumps that force the movement of each of the ions in 
opposite directions thus maintaining a constant charge separation across the mem- 
brane and keeping the cell at its resting membrane potential. A typical value of the 
resting membrane potential is -60 mV measured inside the cell with reference to the 
outside. Using (6.1), the membrane potential associated with each of the ions is: 

E = ^ln^ (6-7) 

zF [X]. 

where z is the valence of the ion. Although the membrane potential is dependent 
upon the ionic fluxes, it is not equal to either of their membrane potentials. Instead, 
the membrane potential of the cell is determined by the concentrations of the ions 
inside and outside the cell along with the ease with which each of ions can cross 
the membrane, i.e. on the conductivity and permeability of the membrane to the 
specific ions. The Goldman equation describes quantitatively the dependence of the 
membrane potential at steady state on ionic concentration and permeability {P): 

v _ RT ln P K JK + 1 + P Na [Na + 1 + P a [CT J (68) 

'" F P K [K + ]. + P Na [Na + l + P a [CI- ],. 

As mentioned previously, the cell membrane is selectively permeable to certain 
ionic species. This is possible due to the presence of ion channels that are pore-like 
structures spanning across the membrane. As an example, the potassium channels 
remain open causing a leak of K + ions out of cell making the inside of the cell more 
negative. During neuronal signaling, the membrane potential rapidly changes in 
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response to some stimulus. This is in part achieved by the reduction of membrane 
potential (depolarization) that leads to the opening of voltage-gated sodium ion 
channels causing a further reduction of membrane potential. Initially the cell's 
response is proportional to the stimulus strength, i.e. the cell responds as a graded 
potential. Once the membrane potential crosses threshold, the cell responds by 
generating an action potential that propagates down the cell's axon all the way 
to its axon terminals. The axon terminals in turn connect to other cells (through 
synapses) thereby activating them and thus initiating a signaling cascade. The 
action potential is described as an all-or-none phenomenon, i.e., once initiated it 
will actively propagate down the axon irrespective of the presence of the initial 
stimulus. Typically, action potentials last for about a millisecond after which the 
cells return to their resting state through the inactivation (closing) of voltage-gated 
sodium channels and activation of voltage-gated potassium channels. These two 
mechanisms have longer time constants compared to sodium activation but work 
together to bring the cell back to its resting membrane potential. This period of 
inactivation is called the refractory period. 

Electrical stimulation of excitable tissue generates action potentials that in turn 
initiate neuronal signaling and enable partial restoration of lost functionality in 
sensory or motor systems. This process requires the extracellular region to be 
driven more negative by applying a rapid negative charge injection via an extracel- 
lular stimulating electrode. For the simplest case of stimulation, a single electrode 
is placed near the excitable tissue and the electrode is driven as a cathode causing 
the outside of the membrane to become more negative. This causes the membrane 
potential to become positive thus leading to a net reduction in the membrane poten- 
tial (depolarizing the membrane). If on the other hand, the stimulating electrode is 
driven as an anode, then it will cause the outside of the cell to become more positive 
than the inside thus causing the membrane potential to become more negative. This 
will lead to a net increase in the membrane potential causing the membrane to 
hyperpolarize. Since a current generator must have a source and a sink, during 
extracellular stimulation, a second electrode is required for the current loop to be 
complete. This second electrode is usually called the return electrode and based 
upon its size and position can cause a number of different events to occur. If the 
return electrode is much larger than the stimulating electrode, then the current 
density is highest at the stimulating electrode causing excitation of neurons near it. 
However, if the return electrode is similar in size as the stimulating electrode, then 
the current density at both sites will be the same and hence neuronal excitation can 
occur at both sites. In this case, during cathodic stimulation, the neurons in close 
proximity to the stimulating electrode are depolarized while those underneath the 
return electrode are hyperpolarized. In some case this hyperpolarization may 
be large enough to suppress an action potential initiated near the electrode (anodic 
surround block) [41, 43]. On the other hand, if anodic stimulation is employed 
then the neurons near the stimulating electrode will be hyperpolarized while those 
near the return electrode will be depolarized. In this case, the action potentials 
are initiated in regions distant from the electrode known as virtual cathodes. 
The depolarization that occurs through anodic stimulation is about a seventh to a 
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third of that accomplished through cathodic stimulation although this depends upon 
the electrode position [44]. Thus, cathodic stimulation requires less current to cause 
a cell to cross threshold and initiate action potentials. 

The stimulation protocols described above may be effective at selectively acti- 
vating one population of neurons without activating neighbouring neurons. 

Activation thresholds are usually defined in terms of the amount of current 
needed to cause the excitation along with the duration that the current is applied. 
Another way to define excitation thresholds is in terms of the applied charge that is 
simply a multiplication of the amplitude of the applied pulse (current) with the 
duration of the pulse. Since in neural stimulation, currents applied are in the range 
of microamps and are applied typically for a few milliseconds, the charge delivered 
ranges from a few microcoulombs to a few nanocoulombs. By far, the best known 
law of stimulation is the one by Lapicque that relates the threshold current (I) 
required for stimulation to the duration (d) of the applied pulse [32]. He introduced 
the tissue specific excitability parameter called the chronaxie (c) and defined it as 
the pulse duration that required twice the rheobase current (b). Here, rheobase 
current is defined as the threshold current (I) for very long pulses. Mathematically, 
b is the limit of I, as pulse duration goes to infinity. The Lapicque law for stimula- 
tion is: 

I = b{\ + cld) (6.9) 

Based on the above equation, strength-duration curves can be plotted to graphi- 
cally illustrate the relationship between the three parameters I, d and c, as shown in 
Fig. 6.4. The strength-duration curve is an essential tool in all types of studies 
where electrical stimulation of excitable tissue is employed. Studies have shown 
how different parameters can be calculated from these curves including charge and 
energy-duration relationships [24]. Although numerous studies illustrate chronaxie 
values of different excitable tissues, the accuracy of the measurements can be 
affected by factors such as the electrode characteristics, tissue inhomogeneity, 
stimulus waveform, etc. [22, 23]. Studies in motor nerves and different types of 
muscle have shown the dependence of chronaxie on different parameters such as 
temperature and location of electrodes [22]. 

Another way to define the relationship between stimulus strength and excita- 
tion is through amplitude-intensity function, as shown in Fig. 6.5. This is typically 
used where the response is an evoked potential and generates a plot of the stimu- 
lus strength at fixed pulse duration against the amplitude of the evoked response. 
It helps in determination of true threshold by simply extrapolating the curve to 
intersect the x-axis. Amplitude-intensity functions are useful because neural 
prostheses typically operate above threshold to provide a range of sensation or 
activation. Finally for the case of single units, analysis methods such as post- 
stimulus time histograms (PSTHs) are employed that sort the individual spikes 
based on their latencies. More sophisticated analyses of a mixture of action 
potentials produced by multiple cells involve grouping the individual spikes 
based on their individual waveform characteristics. 
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Fig. 6.4 Strength-duration graph illustrating threshold current required to elicit response at different 
pulse durations. Rfieobase current = b; chronaxie = c. Modified from [23] 
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Fig. 6.5 Representative graph illustrating the gradual increase in response amplitude as the 
stimulus strength is increased. The amplitude of response is usually measured in microvolts (uV) 
while the applied stimulus amplitude is usually in microamps (uA) 

6.6 Safe Stimulation of Tissue 



A neural stimulation system that is not properly designed can cause damage to the 
tissue or to the electrode itself. For any neural stimulation system to be successful, 
it must elicit the required neuronal excitation without causing any damage to the 
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biological system. Electrode shape, size and material along with stimulus pulse 
parameters need to be carefully chosen to meet the requirements of the system. 
Extensive work has been done in defining the role of all the different parameters 
that determine the safety limit of the tissue and electrode. 



6. 6. 1 Mechanisms of Neural Injury 

There are several mechanisms that may cause neural injury; they are broadly catego- 
rized into two main classes. The first mechanism of damage is associated with the 
electrochemical processes through which the stimulus current is injected into the 
target tissue. Damage is induced due to formation of toxic electrochemical reaction 
products during stimulation at a rate greater than what can be tolerated by the physi- 
ological system. These damaging processes have been well characterized using elec- 
trochemical methods as discussed in Sects. 6.2 and 6.3. A second mechanism of 
neural injury is associated with the flow of current through the target tissue [35]. This 
involves the metabolic stresses induced on the tissue causing a transient or permanent 
elevation of neurotransmitter release (excitotoxic effect). It may also include large 
depolarizations and hyperpolarizations induced by the voltage gradient (membrane 
electroporation). This second mechanism is multi-factorial and complex. 



6.6.2 Parameters for Safe Stimulation 

One of the well-established principles of neural stimulation is to achieve charge 
balancing during stimulation between the different phases of the stimulus pulse. 
This was first reported by Lilly in 1961 and ensures that the total net charge during 
stimulation at the electrode-tissue interface is zero [34]. If charge balancing is not 
accomplished, then a net accumulation of charge will ultimately lead to the rise of 
electrode potentials to levels where water hydrolysis will start. For monophasic 
stimulation, charge balancing is accomplished by the use of a blocking capacitor 
that slowly discharges after the application of the pulse. Although charge - balancing 
ensures that there is no net accumulation of charge, it does not guarantee safety. 
Such stimulus waveforms may momentarily exceed the established safety limits of 
total charge, charge density or electrode potential. Classically, safety limits for 
neural stimulation have been divided into two broad categories: 

1 . Neural damage limits dictated by the ability of biological tissue to withstand 
electric current without any degradation. 

2. Electrochemical limits based on the ability of the electrode to store or dissipate 
electric charge without exceeding the water window, outside of which formation 
of harmful products start. 

While neural injury limits are defined in terms of both charge density and charge per 
phase, electrochemical limits are defined in terms of charge density only. Charge 
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density is simply the total charge per unit area of electrode and determines the 
magnitude of the depolarization or hyperpolarization induced in the neurons and 
axons close to the electrode. Charge per phase is the amount of charge injected during 
each phase of the stimulus pulse and determines the distance over which the applied 
stimulation can activate the neurons, i.e. the number of neurons activated. McCreery 
et al. [37] have shown that charge density and charge per phase act synergistically to 
determine the safe or unsafe levels of stimulation. They showed that neural damage 
is induced with low charge per phase but high charge density, as is often the case for 
microelectrodes. Based on these data delineating the boundary between safe and 
unsafe charge injection for different charge and charge density levels, Shannon et al. 
[51] developed the following empirical relationship: 



log(D) = *-log(e) 



(6.10) 



where, D is the charge density in uC/cm 2 /phase and Q is the charge per phase in 
uC/phase. The equation describes a family of lines for different values of k. The 
line for which k=1.5 describes combinations of charge density and charge per 
phase values for which no damage was observed. Merrill et al. have graphically 
summarized the work of both studies and also included results of other studies 
assessing safety of neural stimulation (Fig. 6.6). 

Along with charge density and charge per phase, other stimulus parameters such 
as frequency of stimulation, duration, etc. have been found to play an important role 
in determining the presence or absence of neural damage. McCreery et al. [38] 
demonstrated the effect of stimulus frequency as a parameter in causing injury during 
peripheral nerve stimulation. Their study showed that continuous stimulation of the 
cat sciatic nerve for 8 h over 3 days causes the myelin sheath to collapse into the 
axonal space leading to early axonal degeneration (EAD). The threshold of neural 
injury decreased with increasing stimulus pulse frequency (Fig. 6.7). 
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Fig. 6.6 Charge (Q) vs. charge density (Q/A) for safe stimulation. Different symbols indicate 
results of different studies. Reprinted from [39], with permission 
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Fig. 6.7 Percentage of myelinated axons undergoing degeneration 7 days after undergoing 8 h of 
continuous stimulation. At higher stimulus frequency, the percentage of axons undergoing EAD 
can be substantial even at low stimulus current. Reprinted from [38], with permission 

Most of the aforementioned studies have employed single electrode stimulation. 
However, a recent study [36] found that in the case of multi-electrode stimula- 
tion, both sequential and simultaneous stimulation at levels previously found to 
be safe create transient depression in the resulting neural response. One theory 
put forward by the authors is the creation of overlapping electric fields that cause 
certain neurons to be driven at rates higher than what is actually being delivered. 
The authors dubbed the observed effect "SIDNE" (stimulation induced depres- 
sion in neuronal excitability). 



6.6.3 Stimulation Induced Injury in the Retina 



To date, most safety studies have been carried out in structures such as the cortex, 
muscle, etc. With increasing efforts towards developing retinal implants [20, 56, 59], 
extensive studies are being done to understand the response of the visual system to 
artificial stimuli [2, 29, 49, 50]. However, only a few studies so far have been dedi- 
cated towards understanding the consequences of long-term stimulation. Giiven 
et al. [27] carried out chronic stimulation studies in dogs and found that the retina 
is able to tolerate chronic stimulation at 0. 1 mC/cm 2 without any histological 
detectable damage or change in the electroretinograms (ERGs). Another study 
investigated chronic stimulation effects through suprachoroidal-transretinal stimu- 
lation [40]. The results of the study showed that threshold for safe charge increased 
logarithmically or almost linearly with increasing stimulus duration but the threshold 
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for safe current decreased logarithmically with increasing stimulus duration. There 
was severe damage in the inner layers when the applied current exceeded this 
threshold. Colodetti et al. [11] found that the retina is sensitive to pressure exerted 
by the electrode. They studied the type of damage due to pressure exerted by the 
electrode with and without accompanying high charge stimulation in the rodent 
retina. Although the type of damage exhibited in both cases were roughly similar, 
the extent of damaged area was significantly larger in the case of accompanying 
high charge stimulation. These studies although informative do not in any way give 
a complete picture of how the retina would respond to continuous stimulation. 
Also, as increasing efforts are being made to make these implants more sophisti- 
cated, the added requirement of a large number of closely spaced electrodes makes 
it imperative to study the possible consequences of high level stimulation on both 
the retina and associated cortical structures. Recent work in these areas is presented 
in Chaps. 7 (Loudin, Butterwick, Huie and Palanker) and 12 (Fried and Jensen). 
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Chapter 7 

Delivery of Information and Power 

to the Implant, Integration of the Electrode 

Array with the Retina, and Safety of Chronic 

Stimulation 

James Loudin, Alexander Butterwick, Philip Huie, and Daniel Palanker 



Abstract The fundamental function of a visual prosthesis is to deliver information 
about a patient's surroundings to his/her neurons, usually via patterned electronic 
stimulation. In addition to transmitting visual information from the outside world 
to the implanted stimulating array, visual prostheses must also pass the electrical 
power necessary for such stimulation from the external world to the intraocular 
electrode array. The first section of this chapter reviews three common methods 
for achieving this data and power transfer: direct wireline connections (suitable for 
research studies), inductively coupled coils, and photodiode-based optical systems 
which utilize the natural optics of the eye. 

Once the data and power has been received, retinal prostheses must effectively 
deliver stimulation currents to surviving retinal neurons. This necessitates an under- 
standing of the electrode/retina interface. The second section of this chapter is a 
histological description of this interface for the case of subretinal implants, investi- 
gating the tissue response to flat implants coated with different materials. Several 
three-dimensional geometries are also described and evaluated to decrease the 
implant-neuron distance. 

Finally, stimulation currents must not damage the stimulated neurons. The third 
section of this chapter describes measurements and scaling laws associated with 
tissue damage from electric currents. Damage thresholds are found to be approxi- 
mately 50-100 times stimulation thresholds. 
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AC Alternating current 

ASR Artificial silicon retina, a retinal prosthesis fabricated by Optobionics 

CMOS Combined metal on silicon 

CMP Computational molecular phenotyping 
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DC Direct current 

EU European Union 

IMI Intelligent medical implants, a company fabricating a retinal prosthesis 

INL Inner nuclear layer 

IR Infrared 

LCD Liquid crystal display 

MPDA Microphotodiode array, retinal prosthesis fabricated by retina implant 

AG 

ONL Outer nuclear layer 

P45 45 days after birth 

PI Propidium iodide 

RCS rat Royal College of Surgeons rat, a common animal model of retinal 

degeneration 

RF Radio frequency 

RPE Retinal pigmented epithelium 

SIROF Sputtered iridium oxide film 

SU-8 A photo-curable epoxy 

USC University of Southern California 



7.1 Introduction 



One of the fundamental challenges for a visual prosthesis is to efficiently deliver 
visual stimuli from the external world to target neurons in the retina, optic nerve, 
or visual cortex. Power and visual information must be transmitted and subse- 
quently distributed over an electrode array while ideally not interfering with 
residual vision, and keeping the natural association between visual information and 
eye movements. Four basic methods have been used to achieve this: direct wireline 
connection to implanted stimulators, radio frequency (RF) telemetry, serial optical 
telemetry, and parallel optical telemetry. In the first part of this chapter we review 
these techniques in their various incarnations. 

After the data is received, providing the appropriate stimulus to the retina pres- 
ents a new set of challenges: high-resolution prostheses require that nearby neurons 
are stimulated with high selectivity and broad dynamic range. While the electric 
field created by the electrode array and the constraints on cellular proximity have 
been characterized [35], the process of maintaining this proximity between elec- 
trodes and cells is less understood. Chronically preserving apposition between an 
epiretinal prosthesis and neurons requires only mechanical stabilization of the 
implant in the vitreous cavity. However, doing so with a subretinal prosthesis 
requires controlling the response of the retina to an implant. In the second part of 
this chapter we describe techniques used to mechanically stabilize implants, and the 
response of the retina to various implant geometries and coatings. 

One of the critically important issues in development of retinal prosthesis is 
understanding the safe limits of electrical stimulation for prolonged periods of time. 
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In the third part of this chapter we describe the dependence of the damage threshold 
on pulse duration, electrode size and its separation from the cells, as well as on the 
number of pulses. We also compare damage and stimulation thresholds to assess the 
safe therapeutic range at various pulse durations. 



7.2 Power and Data Transmission 
7.2.1 Wireline Connection 

William Dobelle led one of the earliest attempts at constructing a visual prosthesis. 
In a series of studies begun in 1968, he used direct wireline connections to link 
electrodes placed in the visual cortex with a stimulator worn externally to the body 
[16]. Subsequent electrical stimulation successfully evoked visual responses in 19 
blind patients, offering hope that future prostheses would one day restore some 
degree of useful sight. 

Direct percutaneous connections are far from ideal, as they can provide pathogens 
with a direct pathway through the skin and are prone to severe scarring [30]. 
Despite this, transdermal cables have often been used in short-term human trials 
of various visual prostheses [51, 53, 65, 76], because of the unrivalled electrical 
versatility which they offer. For example, a group at the Naval Research Lab has 
developed a 3,200 electrode epiretinal prosthesis which is driven with a cable con- 
taining ten wires [58]. This prosthesis is intended for acute experiments; a future 
version under development is wireless. In at least one case, percutaneous cables 
driving a retinal prosthesis have been left in place for a period exceeding 1 year 
[76]. Though direct connections will likely continue to be used in research settings 
for years to come, any future commercial prosthesis will be wireless. 



7.2.2 Inductive Coils 

Inductively coupled coils are used for wireless data and power transmission in a 
wide variety of applications, including medical implants such as cardiac pacemakers 
[4] and cochlear prostheses [74]. More recently, the unique power and data require- 
ments of visual prostheses have spurred much research in the field, with inductive 
coil systems currently developed for epiretinal [33, 69], subretinal [35, 63], visual 
cortex [64], and optic nerve stimulators [65]. 

In all of these designs, an AC current driven through an external transmitting coil 
induces an AC voltage on an implanted coil, which is converted to DC power by 
implanted circuitry. Sometimes the transmitter encodes data onto this signal, which 
is also recovered by the implanted circuitry. Since the coils are only weakly coupled 
to each other (typical values for coupling coefficient k are in the range 0.08-0.24 
[68], compared to -0.9 for standard transformers), great care must be taken to 
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optimize the receiving circuitry. With this in mind, a capacitor is added in series with 
the receiving coil to create a tuned resonance at the transmitter frequency, f. The 
resulting circuit amplifies the received voltage by the quality factor Q, typically in 
the range 10-100. High Q values yield more efficient power transfer thus helping 
to decrease the body's exposure to radiation. The optimization of coil geometry and 
receiving circuitry to maximize Q has been the subject of numerous studies [19, 20, 
27, 28, 31, 62, 66, 68, 72]. Since Q is proportional to the transmission frequency, 
high frequency operation yields higher Q values; however, tissue's RF absorption 
increases exponentially beyond a few MHz [43] limiting transmission frequency / 
to 1-10 MHz. 

Inductive coils have been used to deliver data to visual prostheses for over half 
a century. In the 1960s, a team led by Giles Brindley of the Medical Research 
Council in London implanted an array of 80 coils beneath the pericranium of a 
blind patient [8]. The 80 coils were connected through separate rectifying circuits 
[7] to 80 platinum electrodes placed onto the surface of the patient's visual cortex. 
Individual electrodes were activated by placing a transmitting coil on the scalp 
directly above the electrode's receiver. Interference was minimized by tuning adja- 
cent receivers to different frequencies. Though this scheme was rather successful 
(of the 80 electrode placements, 39 elicited phosphenes), it is hardly scalable. With 
the goal of scaling visual prostheses to hundreds and eventually thousands of pixels, 
higher data rates must be extracted from fewer coils. 

Ironically, while high-Q coils are efficient power receivers, they are rather poor 
data receivers. According to the Shannon-Hartley theorem [59], the data capacity 
C of a coil may be expressed as 

f 
C=B-\og 2 (l + SNR) = —log 2 (l + SNR) C 7 - 1 ) 

where C is in bits per second, B is the bandwidth of the receiving circuit, / is the 
transmission frequency in Hz, and SNR is the signal to noise power ratio. Thus, 
while received power is directly proportional to coil Q, the attainable data rate is 
inversely proportional to Q. For this reason, many visual prosthesis designs use two 
coil pairs: one for power, and one for data, where data transmission is accomplished 
at a higher frequency [27, 68] or with a lower-<2 coil [35]. In addition, complex 
single-coil systems capable of delivering both power and data over one coil pair 
have also been developed [18, 19, 67], in one case achieving a data rate in excess 
of lMb/s[33]. 

Ignoring the time involved in implant monitoring feedback signals, transmitting 
control signals, and other housekeeping functions, the maximum number of pixels 
N that can be individually driven at the refresh rate, R, is determined by the data 
rate, C, and the number of the stimulation strength levels S, as 

N = (7-2) 

*-log 2 CS) 
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For example, the system presented in [33] with a data rate of 1 Mb/s, refresh rate 
of 60 Hz, and 16 different stimulation levels can adequately support more than 
4,096 pixels (a 64 by 64 array). 

Alternatively, James Weiland's group in USC is developing an implantable 
intraocular video camera [69]. Such a design would do away with the need for 
inductive data transmission altogether, and will require only power and (low band- 
width) control signals. 

Power and data transmission efficiency are inextricably intertwined with the 
physical coil placements. As such, the ideal surgical placement for receiving coils 
is a matter of ongoing debate. In Second Sight's first generation prosthesis, two 
coils implanted subcutaneously behind the ear were coupled to a second pair, 
attached to the frame of eyeglasses worn by the patient [23]. This location was 
chosen for two reasons: first, because surgeons have years of experience implanting 
coils behind the ear, due to the success of cochlear implants. Secondly, the outer 
coil may be placed very close to the implanted one, thereby maximizing coupling 
efficiency. The disadvantages are also twofold: first, any operation implanting 
both a retinal stimulator and a posterior auricular coil-set necessitates both a retinal 
surgeon and an otolaryngologist. This increases both the cost and the length of 
the operation. Secondly, a trans-scleral cable must connect the coil to the intraoc- 
ular stimulator. This wire must be thin and flexible enough to allow normal eye 
movement, while also robust enough to withstand years of bending without failure. 
Fabricating such a wire is a challenge, though not an insurmountable one. For 
these reasons Second Sight has changed to a periocular design for their second 
generation prosthesis (Argus II), with receiving coils mounted on the front of 
the eye globe. The transmitting coils are mounted on the front of the patient's 
eyeglasses [2]. 

The Boston Retinal Implant Project has designed a pair of coils and receiving 
circuitry which is sutured to the side of the eye under the conjunctiva, facing 
towards a transmitting coil mounted on the side arm of eyeglasses worn by the 
patient [63]. The German EPIRET consortium attempted a different approach, 
removing the lens and placing the receiving coil in the lens capsule [40]. This 
technique is appealing due to its similarity to cataract surgery. However, the rela- 
tively small size of the lens capsule puts tight constraints on the coil size (approx- 
imately 12 mm). 



7.2.3 Serial Optical Telemetry 

Photodiodes are excellent data receivers. Standard CMOS integrated photodiodes can 
have bandwidths in excess of 1 GHz [49], over two orders of magnitude higher than 
those attainable with inductive coils, whose bandwidths cannot exceed ~10MHz due 
to transmission frequency and Q-factor constraints (see (7.1)). The German company 
IMI Technologies is developing an epiretinal prosthesis with a subconjunctivally 
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placed inductive coil and a photodiode placed inside the vitreous cavity [69]. The coil 
receives power inductively, while the photodiode receives stimulation data optically. 
As in the case of inductive coil systems, the data is delivered serially over one channel, 
so it must be decoded and distributed over the electrode array using a data processing 
chip connected to the retinal array by intraocular cable. 



7.2.4 Photodiode Array-Based Prostheses 

The Chow brothers were the first to propose and investigate the use of photodiodes 
in a retinal prosthesis, in the early 1990s [12]. Optobionics, a company they 
founded, developed a two dimensional array of 5,000 photodiodes on a 2 mm silicon 
disk. These so-called artificial silicon retinas (ASRs) were fabricated such that all 
5,000 photodiode anodes were connected together, while their cathodes were elec- 
trically isolated from each other [48]. Each cathode contained its own iridium oxide 
electrode, resulting in a 2mm device with 5,000 light-controllable electrical 
sources. Since each photodiode in the array collected light simultaneously, visual 
data could be delivered to all 5,000 pixels in parallel. This is in contrast to the serial 
RF and optical telemetry systems described above. 

The Optobionics design assumed that ambient light would be directly converted 
by the photodiodes to currents strong enough to stimulate surviving retinal neurons. 
Indeed, initial human trials did result in some vision improvement [13]; however, 
this improvement was not due to electrically-elicited action potentials, but from 
neurotrophic effects resulting from ASR implantation [46]. Unfortunately, ambient 
light intensities provide insufficient current to directly stimulate nerve tissue, by at 
least three orders of magnitude [45]. In addition, since the electrode-electrolyte 
interface is capacitive when driven in a biologically compatible way, a photodiode 
array should be driven by pulsed, rather than continuous illumination. Indeed, later 
in vivo experiments on RCS rats with subretinally implanted ASRs successfully 
demonstrated neural activity in the superior colliculus in response to intense pulsed 
infrared illumination of the retina [15]. 

Eberhart Zrenner and his group from the University of Tuebingen, Germany 
have constructed a photosensitive array equipped with built-in differential amplifiers, 
called the microphotodiode array prosthesis (MPDA) [77]. Each pixel contains a 
photodiode and active circuitry which measures the difference between local and 
global brightness, and then drives a current corresponding to this difference through 
an associated microelectrode. The system is capable of utilizing almost all the elec- 
trochemical capacity of the electrodes; however, it requires separate power delivery 
to drive the active circuitry. So far, short-term human trials have relied on percuta- 
neous wireline connections to deliver this power; future, wireless implants will use 
an inductive coil system for this purpose [6]. Recent result with a human patient 
confirmed the ability of the 1,500 pixel array to provide visual acuity on the order 
of 20/1,000, allowing a patient to read large fonts [53]. 

Daniel Palanker's group at Stanford University has taken a different approach 
to an active photodiode array system by using video goggles to project pulsed 
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Fig. 7.1 Average stimulation current produced by one, two, and three photodiodes connected in 
series. The diodes were oriented such that the biphasic stimulation pulses were anodal-first. Data 
taken for 25 Hz, 500 us pulses with a 50 um SIROF active electrode coupled to a much larger 
return electrode 



infrared (905 nm) images onto the subretinal array. A single, photovoltaically-driven 
photodiode can only produce up to 0.6 V at physiologically safe light intensities 
[35], a fraction of the 1.4 V electrochemically-safe "water window." By providing 
a pulsed bias voltage and utilizing the diodes in a photoconductive rather than 
photovoltaic manner, they can produce bi-phasic currents sufficient for neural 
stimulation, and limited only by the electrode charge injection capacity [35]. The 
common photodiode bias is provided by a periocular coil-based system. Recently 
Palanker's group proposed the use of series photodiodes to receive sufficient current 
photovoltaically [34]. The voltage increase afforded by series photodiodes, combined 
with the nonlinear electrochemical capacitance of iridium oxide electrodes [14], 
greatly increases the attainable current, as shown in Fig. 7.1. Since pulsed infra- 
red illumination is directly converted into electric currents sufficient for stimula- 
tion, there is no need for a wired connection to a separate power-receiving 
module. The pixels do not even need to be physically connected to each other. The 
arrays may be separately placed into the subretinal space, greatly simplifying surgery. 
The information transfer rate C from goggles with N pixels operating at S levels 
of gray at frame rate R can be estimated in a manner similar to (7.2). With an XGA 
LCD display (N= 1,024x768) operating at 25 Hz and 128 levels of gray, the data 
rate is C = 138 Mb/s. The limit in this approach is clearly on the receiving end of the 
system - the photodiode array and its interface with the retina. 



7.2.5 Thermal Safety Considerations 



Power losses due to tissue absorption and intrinsic imperfections in the receiving 
circuitry lead to heating. For coil systems this includes absorption of RF radiation 
in tissue between the transmitting and receiving coils, resistive losses in the coils 
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themselves, and losses in the rectifying circuitry. In photodiode systems this 
includes light absorption in ocular pigments such as melanin and in the implant 
itself. In both system types, the resulting tissue heating must be understood and 
controlled to within acceptable safety limits. 

Tissue RF-absorption has a strong frequency dependence, increasing exponen- 
tially beyond a few MHz [43]. However, power transfer efficiency also increases 
with frequency due to the linear increase of the quality factor Q. There is an optimal 
frequency region balancing these counteracting effects where RF tissue exposure is 
minimized. Most coil designs operate at a frequency between 1 and 10MHz [19, 
33, 67]. Once a frequency is chosen, there exist design methodologies to maximize 
receiving circuit efficiency [27, 28]. Such systems can have power- transfer efficien- 
cies exceeding 65% [28]. 

Photodiodes are rather inefficient at converting light into electrical power. A 
photodiode's maximum conversion efficiency (ratio of current output to incident 
light) typically does not exceed 0.6 A/W. Since photodiodes produce a photovoltage 
of at most 0.5 V at physiologically safe light intensities [35], 1 W of incident light 
power cannot produce more than 0.3 W of electrical power - an efficiency of at 
most 30%. Thus, photovoltaic retinal stimulation is a rather energy intensive task, 
rendering it imperative to examine safety limits for intense retinal illumination. 

According to established ocular safety standards [3, 60], the maximum permis- 
sible retinal irradiance for prolonged exposure to near-IR light is 2.8mW/mm 2 . 1 
Similar thermal considerations apply to heating of the iris. Peak irradiance can 
significantly exceed the average value during short pulses if the duty cycle is 
decreased. For example, in a goggles-based system with 1 ms pulses delivered at 
25 Hz, the duty cycle is 1/40. The peak irradiance during the pulse can then be 
increased by a factor of 40 - to 1 12 mW/mm 2 . Assuming a light-to-current conver- 
sion efficiency of 0.4 AAV, the maximum current that can be produced by photo- 
diodes with this irradiance is 45mA/mm 2 , corresponding to a charge density of 
45 |iC/mm 2 . This value exceeds the retinal stimulation threshold on large electrodes 
by at least three orders of magnitude [24]. 

Most retinal-heating studies to date have been acute; little data is yet available 
on the effects of chronic retinal heating. However, it has been observed that chronic 
lens heating by 2-3°C can lead to cataract formation [57], in which case chronic 
heating due to electronic implants could also cause cataracts. A "less than 1°C" 
criterion for implantable devices is codified in EU safety regulation [1] since this 
level is comparable to natural variations of the body temperature [50]. For a disk- 
shaped heater of diameter D which dissipates power P the maximum temperature 
rise AT in the adjacent medium is [52] 



AT: 



AXD 



(7.3) 



'The ED50 level for producing a minimally- visible lesion with near-IR light (X= 8 10-950 nm) for 
spot sizes larger than 1.7 mm on the retina and exposure times exceeding 1000s is 56mW/mm : 
[42, 43]. With a safety factor of 20, the maximum permissible exposure is then 2.8mW/mm 2 . 
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where X is the heat conductivity of the medium. For example, keeping temperature 
rise under 1°C for a 1 cm disk in water (X = 0.58 WK" 1 m -1 ) requires dissipating less 
than 23 mW of power. Blood perfusion in the eye helps to cool the tissue more 
efficiently, especially at temperature rises exceeding 3°C [55, 56]. 



7.2.6 Conclusions: Comparing the Different Approaches 

Though the drawbacks of percutaneous connections are many, they will continue to 
be used in research environments through the foreseeable future. The electrical 
access afforded by percutaneous cables is invaluable when studying stimulation 
thresholds or reading the changing electrical properties of tissue. In addition, the 
development of a wireless system can be quite challenging - direct connections can 
greatly simplify experiments. However, any commercial prosthesis will need to 
incorporate a wireless power and data delivery system of some sort. 

Of the three wireless system types described (RF, serial optical, and parallel 
optical telemetry), wireless cortical and optic nerve prostheses exclusively use 
radio links [8, 64, 65]. For retinal prostheses, there is as of yet no clear answer as 
to which system is superior. The circuit complexity of inductive systems is typically 
much higher than for photodiode-array based ones. Data signals must be separated 
from the carrier frequency, decoded, stored, and routed from the coil to individual 
electrodes. Photodiode array pixels receive all data simultaneously, with no need 
for complex wiring schemes. In addition, photodiode systems can produce DC volt- 
age directly, whereas inductive coils produce AC current which must then be con- 
verted to DC. However, in terms of power transfer efficiency, inductive coil systems 
have a clear edge over photodiode systems, achieving efficiencies of greater than 
65%, vs. -30% achievable with photodiodes. 

Photodiode-array based prosthetics keep the natural association between eye move- 
ments and visual perception, since they use the eye's natural optics. A shift of the 
patient's gaze directly changes what part of the visual field falls on the photodiode 
array. In contrast, current coil designs, as well as a serial optical telemetry with one 
receiving photodiode deliver visual information based solely on the orientation of a 
head-mounted camera; shifts in gaze do not change the visual stimuli. This could 
be fixed in the future by adding an eye-tracking system which controls what part of 
the visual field is transmitted to the stimulator. The Weiland group's proposed 
eye-implanted camera [69] would also produce images which shift naturally with eye 
movements, although it would add significant complexity to implanted electronics. 



7.3 Tissue Response to a Subretinal Implant 

The sophistication of the visual system requires a prosthetic much more complex than 
previous electrical stimulation devices including pacemakers, cochlear implants and 
deep brain stimulators. Ensuring that electrodes maintain sufficiently close proximity 
to the target cells is an important aspect of interfacing a prosthetic with the retina. 
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Visual acuity of 20/200 (the threshold of legal blindness in the United States) 
geometrically corresponds to a spatial frequency of three cycles per degree or ten 
lines per millimeter on the retina [29]. Since at least 2 pixels per cycle are required 
for appropriate sampling (Nyquist-Shannon sampling theorem), achieving this 
resolution constrains the maximum pixel size to 50 urn, or a pixel density of 
400 pixels/mm 2 . If the electrode is to be no larger than half the size of the pixel, 
then the electrode diameter should not exceed approximately 25 urn. The diver- 
gence of the electric field from the electrode requires greater currents to stimulate 
cells at increasing distances. Since increasing separation also causes the divergent 
electric field to influence larger regions of the retina, this effectively reduces the 
specificity of neural stimulation, resolution and contrast [45]. Charge and current 
injection limits, determined by the electrochemical capacitance of the electrode 
material, also determine the maximum distance at which neurons can be stimu- 
lated. In addition, significant variation in the distance between electrodes and 
neurons across an implant will lead to position-dependent differences in stimula- 
tion thresholds and responses. These factors dictate that the distance between the 
electrodes and target cells should not exceed the electrode size [45]. In the case of 
50 urn pixels with 25 urn electrodes, the separation of electrodes from cells should 
ideally not exceed 25 urn. 

Retinal neurons can be stimulated electrically using arrays of electrodes posi- 
tioned either epiretinally [22, 37, 38] or subretinally [54, 61, 75]. Although surgi- 
cally more challenging, a subretinal array placement to stimulate bipolar cells has 
the potential advantage that the electrical stimuli can be simpler. Since the cells in 
the inner nuclear layer (INL) have a graded response (they do not spike), they may 
not require as precise stimulus timing as the spiking ganglion cells. Subretinal 
electrical stimulation can produce a graded response which is then converted to 
spiking ganglion cell output via natural signal transduction pathways [61]. 
Addressing the visual system earlier in the signal processing cascade may also 
utilize some preserved natural signal processing mechanisms between the inner 
retina and ganglion cells. In contrast, direct ganglion cell excitation with epiretinal 
electrodes bypasses inner retinal circuitry and requires significantly more complex 
signal processing by external elements. 

In the subretinal approach, the proximity between the stimulating array and the 
bipolar cells can be limited by subretinal gliosis and fibrosis, whereas in the epireti- 
nal approach, proximity is limited by the inner limiting membrane and the nerve 
fiber layer. Mounting the implant epiretinally presents an additional challenge: 
attachment via a single retinal tack often results in tens or even hundreds of microns 
of separation between the retina and peripheral parts of the implant [36]. In con- 
trast, the subretinal approach appears to provide more consistent proximity to the 
retina along the implant [47], although the thickness of the degenerating retina may 
be uneven. 

In the following section we present results of investigations on the effect of an 
implant's material and shape on its integration with the retina. 
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SU-8, a photo-curable epoxy, was used to manufacture devices to investigate tissue 
response to subretinal implants. SU-8 polymers are ideal for this purpose because they 
can be coated with materials commonly used in retinal prostheses, can form high 
aspect-ratio structures and are soft enough to be easily sectioned, in situ, on a conven- 
tional microtome to histologically evaluate the implant-retina interface. Three different 
coating materials have been compared: silicon oxide, parylene-C, and iridium oxide. 

The Royal College of Surgeons (RCS) rat is a commonly studied model of retinal 
degeneration and is ideal for subretinal implantations of these devices because of its 
sufficiently large eyes and vascularized retina [32]. All implantations were per- 
formed at 45-60 days of age; at this stage the photoreceptor cells have largely 
degenerated. The implants were placed in the subretinal space using a custom 
implantation tool that protects the implant from mechanical damage during insertion 
[9]. The implant is delivered trans-sclerally through a small incision behind the pars 
plana; the retina is detached from the RPE by injecting BSS with a 30-gauge can- 
nula. Implant placement was evaluated after each surgery by fundus examination. 

Figure 7.2a illustrates the normal wild type rat retina, and Fig. 7.2b shows RCS 
retina at 45 days of age (P45). Figure 7.2c demonstrates a flat SU-8 implant in the 
subretinal space of a P45 RCS rat 6 weeks after surgery. Changes in the degenerat- 
ing retina are easily seen comparing 7.2b to 7.2a; in Fig. 7.2b, the outer segments 
have disintegrated and there is significant thinning of the outer nuclear layer (ONL) 
though the inner nuclear, inner plexiform, and ganglion cell layers are generally 
well preserved. 

As shown in Fig. 7.2c, two types of tissue reaction to the subretinal implant are 
evident: gliosis and fibrosis. Differentiation of retinal pigment epithelial (RPE) cells 
into long fibrotic membranes that encapsulate the foreign body is called fibrosis - this 




Fig. 7.2 Histological sections depicting (a) wild type rat retina, (b) RCS rat retina 45 days post 
natal (P45), and (c) RCS rat retina with a flat SU-8 implant in the subretinal space 6 weeks post-op. 
A fibrotic seal running along the length of the implant is denoted with the left arrow. A region of 
gliosis separating the implant from the INL by 40 \xm is shown by the right arrow. Scale bar is 
50 itm. Figure reprinted from [9], with permission 
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Fig. 7.3 Comparison of typical tissue responses to flat implants with different coatings 6 weeks 
post-op. (a) SiO, coating appears to induce significant fibrosis over the implant, (b) IrOx causes 
a mild gliosis above to the implant, pointed by an arrow, (c) Parylene-c coating allows the INL to 
settle down very close to the implant. The INL is separated from the upper surface of the implant 
by only 15-30 |^m. Scale bar is 50|am. Figure reprinted from [9J, with permission 



dark stained layer apposed to the implant is denoted by an arrow in Fig. 7.2c. The 
lightly stained reaction in the inner nuclear layer (INL), denoted by an arrow in 
Fig. 7.2c, is the hypertrophy of glial cells processes and is called gliosis. Fibrosis and 
gliosis can separate the INL from the surface of the implant by approximately 
40|im. 

The silicon oxide coating (Fig. 7.3a) induced significant fibrosis around the implant. 
Generally, the iridium oxide and parylene coatings (Fig. 7.3b, c) were well tolerated 
with only a mild gliotic response, resulting in 15-30 um separation from INL somata. 



7.3.2 Chamber Implants 

The tendency of retinal cells to migrate into the voids of three-dimensional implants 
can be used to provide closer apposition between the implant and the inner nuclear 
layer [44]. The vertical movement of the retina is hypothesized to be largely due to 
the movement of glial processes into the voids. Retinal neurons appear to be pulled 
along with retinal glia - this movement occurs within 72h after implantation [17]. 
The ability of these structures to maintain a stable interface and suppress severe 
fibrosis is discussed below. 

Chamber structures were fabricated with an array of wells (40 x 40 x 20 |im tall) 
in an SU-8 substrate (Fig. 7.4a, b). These devices were implanted into the subretinal 
space of RCS rats. A representative histological section of the retina, 6 weeks post- 
implantation, is shown in Fig. 7.5. In general, INL cell bodies migrate through 
apertures larger than 20 |im (right three chambers). In some chambers with apertures 
greater than 20 urn a retinal microvasculature developed [9] . 

Computational molecular phenotyping (CMP) analysis was used to distinguish 
superclasses and classes of neurons and glia, determine their status, examine mor- 
phological circuitry changes in response to retinal degeneration, and document 
any influence the implant may have upon these processes [25, 26]. Within the time 



7 Delivery of Information and Power to the Implant 



149 




Fig. 7.4 SEM of three-dimensional implant structures, (a) Two microfabricated layers of the 
SU-8 chamber structures prior to adhesion to the basal membrane. Chamber sizes are 40 and 
20 i^m, and aperture sizes are 20 and 10 u,m. (b) High magnification view of the chamber array. 
The 10 and 20 itm apertures can be seen clearly in the center of the 40|^m chambers, (c) Implant 
with an array of SU-8 pillars at three densities, with center-to-center distances of 60, 40 and 
20 (am. (d) High magnification SEM of the pillar array. Pillars are 10|jm in diameter and 65 u,m 
in height. All scale bars in this figure are 100 \im. Figure reprinted from [9], with permission 




Fig. 7.5 The chamber structure implanted into P45 RCS rat subretinally for 6 weeks. The three 
chambers on the right have 20itm apertures, the two on the left are 10u.m. This is an example of 
a typical section where cell bodies have migrated through the wider apertures while only processes 
migrated through 10 u.m apertures. Artifactual folds are marked with a *. Scale bar is 50itm. 
Figure reprinted from [9], with permission 
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window of this study, up to 6 weeks postoperatively, the neurons maintain their 
normal narrowly defined small molecular signatures in the presence of the implant, 
indicating normal metabolic status [9]. 



7. 3. 3 Pillar Arrays 

While chamber arrays effectively improved neuron-implant proximity, the migrated 
tissue was somewhat isolated from the rest of the retina. Pillar arrays were designed 
to provide proximity to cells by utilizing retinal migration, while avoiding this 
isolation. The pillars used in this study were made from uncoated SU-8, approxi- 
mately 10 urn in diameter, spaced 20, 40 and 60 urn center- to-center and 65 urn in 
height (Fig. 7.4c, d). The devices were implanted into P45 RCS rats for 6 weeks. 

Figure 7.6 shows retinal histology 6 weeks after implantation. In the area with lower 
pillar density (Fig. 7.6a, 40 um spacing) some cell bodies appear to be pulled down past 
the tops of the pillars, but many cell bodies remain apposed to the electrode surface at 
the top of the pillars. At the higher pillar density (Fig. 7.6b, 20 um spacing) the space 
between the pillars seems to be filled almost entirely with neuropil and the cell bodies 
remain stable near the tops of the pillars. The inner retina appears well ordered and 
healthy for RCS retina. CMP results show that the neurons retain their phenotype and 
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Fig. 7.6 Pillar implant in subretinal space of RCS rat 6 weeks post-op. Scale bar is 50 pm. 
(a) Area with 40|am pillar spacing, (b) Area with 20p.m pillar spacing. Figure reprinted from [9], 
with permission 
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function through the migration process, for 6 weeks after implantation, and that there is 
excellent apposition between neuronal cell bodies and electrode tips [9]. 

In summary, the best proximity between electrodes and target cells in the inner 
retina is achieved using three-dimensional implants that utilize retinal plasticity 
for intimate integration of the inner retina with the implant. An implant with a 
multitude of voids allows retinal cell bodies and cell processes to migrate into the 
voids within 72 h after implantation, and appears to be stable at least up to 6 weeks 
post-op. 



7.4 Damage to Retinal Tissue from Electrical Stimulation 

Understanding the safe limits of electrical stimulation of neural tissue is critically 
important for maintaining a stable interface between the retina and the prosthesis. 
Some studies along these lines have been performed with cortical stimulation in 
cats, using charge-balanced 400 |is pulses at 50 Hz over the course of 7h, and ana- 
lyzed by histology [39]. It was determined that charge per phase and charge density 
were cofactors that determined cellular damage in different regimes. However, no 
detailed understanding has been achieved regarding dependence of the damage 
threshold on pulse duration, electrode size, distance from the electrode and number 
of pulses. This section explores these dependences using chick retina, validated 
with a limited number of experiments on mammalian retina in vitro. 

Electrical stimulation was biphasic, with the same duration in both phases, leading 
with the cathodal phase. All durations mentioned below refer to time per phase. Cellular 
damage was assessed using propidium iodide (PI), a normally cell impermeable mole- 
cule that becomes fluorescent upon binding to nucleic acids [70]. PI was added to the 
medium prior to the treatment and dye fluorescence was assessed 15 min after the elec- 
trical pulse. Causes of cellular damage may include the direct effect of electric field, 
thermal damage from the applied current, or toxic products from the electrochemical 
reactions at the electrode-electrolyte interface. Our estimations of Joule heating within 
the pulse durations and currents used in our experiments indicated that this effect is 
negligible - temperature rise did not exceed 0.02°C, significantly lower than thermal 
damage thresholds [10]. Glass pipettes pulled to various tip diameters were used as 
stimulating electrodes. This design allowed large platinum wire bundles inside the 
pipette to have low current density on the metal surface to avoid generation of gas inside 
the capillary, while having high current densities at the electrode tip. 

Damage thresholds were established for pulse durations in the range of 6 |js/ 
phase to 6ms/phase, and for electrode diameters of 0.1-1 mm. All plots include 
two points for each setting: a maximum safe value and a minimum damaging level, 
evaluated 5-15 min after the insult. 

As shown in Fig. 7.7, the damage threshold decreased with the number of pulses, 
stabilizing after 100 exposures at approximately 15% of the single pulse value. This 
level remained stable up to the maximum number of pulses tested - 7,500 expo- 
sures at 25 Hz. The pulse duration was 600 |is in these measurements. 



152 



J. Loudin et al. 



1 


• 










Density 
o 

CO 










■ 


Current 
o 

CO 










■ 


Normalized 
o o 




Kt_ 


• 


t 


7,500 Pulses 










• t 



10 



100 
Number of Pulses 



1000 



15 



10.000 



Fig. 7.7 Retinal damage threshold current density as a function of the number of pulses, applied 
during 5 min, normalized to a single exposure damage threshold. After approximately 50 pulses, the 
damage threshold reaches a constant level. A pipette of 1 mm in diameter was used in these measure- 
ments with a 600 us pulse duration. Figure reprinted from [10], with permission; © 2009 IEEE 
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Fig. 7.8 Strength-duration dependence of the damage thresholds on the retina, (o) Measured on 
the chick retina with single shots (open symbols) and with sustained repetitive exposures (• solid 
symbols). Current density relates to pulse duration f roughly as r 05 , which is characteristic of 
electroporation [69, 70]. For comparison, (x) represents the damage thresholds of the porcine 
retina by single pulses in vitro and (A) presents chronic damage thresholds on the rabbit retina 
measured in vivo [13]. Figure reprinted from [10], with permission; © 2009 IEEE 

7. 4. 1 Effect of Pulse Duration 



Pulse duration was varied between 6u.s and 6 ms using a large electrode (1mm) 
for single and repeated exposures. As shown in Fig. 7.8, the damage thresholds 
scale with pulse length as approximately 1/Vt, or more exactly t" 048 for chronic 
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stimulation (7,500 pulses), and t -041 for single exposures. Two additional measurements 
have been performed with single exposures on porcine retina to validate the chick 
model; they are presented with the X. For comparison we also plot the in vivo result 
of chronic retinal stimulation in rabbits (A) [71]. 

The approximate scaling of the strength-duration curves as t"° 5 is characteristic of 
electroporation [41, 42], indicating that cellular damage is produced by the opening 
of pores in the cell membrane. Though cells can recover from the transient occurrence 
of these pores, it is unlikely that the cell would be able to sustain this abnormal state 
chronically. The scaling also indicates that neither charge, nor charge density, q=j-t, 
are conserved along the strength-duration curve. It used to be believed that charge 
and charge density per phase were the two determinants of damage threshold [39]. 



7.4.2 Electrode Size 

The dependence of damage threshold on electrode size was investigated using 
600 |is biphasic pulses on chick retina with electrode size ranging from 0. 1 to 1 mm. 
As shown in Fig. 7.9, damage threshold current density is nearly constant with large 
electrodes (diameter greater than 300 |im). With smaller electrodes the current 
increases, asymptotically approaching a 1/d 2 dependence, indicating a constant 
current regime characteristic of a point source. This asymptotic constant current 
value was about 140 uA for a 600 |is pulse duration. 

The strength-duration relationship for large and small pipettes (0.115 and 
1.0 mm) are compared in Fig. 7.10 with the retinal stimulation thresholds published 
by Jensen et al. [24]. The inset plot depicts the safe dynamic range of retinal 
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Fig. 7.9 Dependence of the threshold current density on pipette diameter for sustained exposures 
on chick retina with pulse duration of 0.6ms/phase. The solid line represents the current density 
at the tissue, calculated using the model of a disk electrode separated from the retina by 125 |^m. 
On electrodes smaller than 200 jim, the current density scales as 1/d 2 , corresponding to a constant 
current of 139 |J.A. Figure reprinted from [10], with permission; © 2009 IEEE 
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Fig. 7.10 Dependence of the chronic retinal damage threshold on pulse duration measured with 
pipettes of 0.12 (•) and 1.0 mm (o) in diameter. For comparison, we plot stimulation thresholds 
of the retinal ganglion cells measured by [44] using disk electrodes of similar sizes: 0.12 (+) and 
0.5 (x) mm in diameter. Ratios of the damage thresholds to the stimulation thresholds are shown 
in the insert for both electrodes. Figure reprinted from [10], with permission; © 2009 IEEE 

stimulation (the ratio of the damage threshold to stimulation threshold) as a 
function of pulse duration for both electrodes. The maximum (on the order of 100) 
of these curves occur near chronaxie for both electrode sizes. It is important to note 
that although the damage and the stimulation thresholds are dependent on electrode 
size, their ratio, which determines the dynamic range of safe stimulation, appears 
to be practically size independent. 

Comparison of the recent measurements of the stimulation threshold in humans 
(electrode size 0.4 mm, 1 ms, 0.01 A/cm 2 ) [36] and the in vivo damage threshold in 
rabbits (electrode size 0.4 mm, 1ms, 0.46 A/cm 2 ) [71] results in a slightly lower 
ratio, 46. A safe dynamic range of 50-100 is sufficiently broad to cover the linear 
response range of neural cells (typically 10-30 [5, 73]), and is therefore adequate 
for the purpose of prosthetic vision. 



7.5 Concluding Remarks 



The development and testing of retinal prostheses by multiple groups throughout 
the world is rapidly advancing. The delivery of a vast amount of information and 
sufficient power to the retinal neurons has proved to be technically challenging, and 
has required the development of new technologies in many disparate fields. 
Sophisticated coil systems have been developed to transmit and receive power and 
data; others have developed novel optical approaches for serial and parallel data 
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delivery. In both approaches care has been taken to avoid thermal damage to 
surrounding tissues in the process of power transmission. 

Once received by the implanted prosthesis, power and data must be delivered to 
target neurons, a task which requires close neuron-electrode proximity. Many 
materials have been tested to characterize tissue response to the implanted devices. 
In addition, three-dimensional subretinal arrays have been developed to utilize 
retinal plasticity to achieve intimate proximity between neurons and stimulation 
sites. Finally, electrical damage thresholds have been carefully measured to 
characterize the safe dynamic range of stimulation. 

Despite the incredible advancements made in recent decades, there is much left 
to be done. This includes implementation of already proposed ideas, and improve- 
ments to the currently used approaches. Higher resolution implants will allow for 
more sophisticated evaluation of prosthetic vision and will most probably generate 
a need for development of more advanced signal processing algorithms. The past 
two decades of research have been very fruitful - several prosthetic technologies 
are currently being tested in human trials [6, 11,21, 53]. The results from the current 
trials are eagerly awaited by researchers around the world, as they will likely dictate 
the direction of technological development for the next decade. 
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Chapter 8 

Retinal Cell Excitation Modeling 

Carlos J. Cela and Gianluca Lazzi 



Abstract As the electrode density of implantable retinal prosthesis increases, 
simulation becomes a valuable tool to characterize excitation performance, evaluate 
implant electrical safety, determine optimal geometry and placement of implant 
current return, and understand charge distribution due to stimulation. To gain 
an insight into the effectiveness of a retina stimulator, quasi-static numerical 
electromagnetic methods can help estimate current densities, potentials, and their 
gradients in retinal layers and neural cells. Detailed discrete three-dimensional 
models of the retina, implant and surrounding tissue can be developed to account 
for the anatomical complexity of the human eye and appropriate dielectric proper- 
ties. This chapter will cover the basics of quasi-static methods that can be used for 
this purpose. Specifically, authors will focus on the admittance method, the output 
it produces, and possibilities it offers to determine the potential effectiveness of 
a retinal stimulator, ranging from evaluating the current density magnitude in the 
ganglion cell layer, to calculating local activation function in the areas targeted by 
the electrical stimulation. 
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8.1 Introduction 
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Retinal implants can help partially restoring vision to patients suffering from 
degenerative diseases of the retina. Age-related macular degeneration and retinitis 
pigmentosa by replacing the functionality of no longer working photoreceptors 
with systematic electrical stimulation to neural cells further down the optical neural 
path [8, 12]. 

Clinical trials show that electrical stimulation using epiretinally implanted elec- 
trodes causes the appearance of localized white or yellow round phosphenes [12]. 
These percepts must correspond to excitation of cells in the ganglion cell layer (GCL) 
or deeper in the retina, as only these cells map to a location under the stimulating 
electrode; the more superficial nerve fiber layer (NFL) is formed by axons of GCL 
neurons going towards the optic nerve that belong to ganglion cells away from the 
stimulating point (Fig. 8.1). Epiretinal electrode arrays having 16 (4x4) electrodes 
have already been successfully implanted in clinical trials, and efforts are ongoing to 
increase the resolution of the implant; versions with 60 electrodes are currently under- 
going the FDA approval process and 240 and more electrodes are being worked on. 
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Fig. 8.1 Diagram of a transverse cut of retinal model with epiretinal implant close to its surface 
(not to scale). The retina geometry has been approximated as flat in this model. The epiretinal 
implant electrically stimulates the ganglion cell layer (GCL) across the nerve fiber layer (NFL) by 
injecting electrical current using charge-balanced biphasic pulses. The NFL is formed by the 
axons of cells in the GCL, which curve and bundle together, eventually shaping into the optic 
nerve, which relays the visual signal to the brain. In all layers but the NFL, the neural pathway is 
predominantly vertical. In this configuration the 25 electrodes are arranged in a regular 5x5 
matrix and partially embedded in a dielectric substrate. A three-dimensional view of the implant 
model is shown in Fig. 8.2 



Retinal Cell Excitation Modeling 161 




Fig. 8.2 Geometry of the 5x5 electrode array used in the model (not to scale). The electrode 
array is positioned inside the ocular globe, in close contact with the retina. In the model simulated 
the electrodes are 10 nm away from the retinal surface, and the gap in between is filled with vitreous 
humor. The electrodes are encased in a block of a dielectric. The current return is positioned on 
the back of the assembly, and exposed to the vitreous humor, which is a relatively good electrical 
conductor 



As arrays become denser, electrodes are packed closer and have smaller size. 
There is a compromise between electrode size, stimulation rate, and charge injected. 
A smaller electrode must use higher current densities to inject the same amount of 
charge in the same time compared to a larger electrode. Current densities must be 
limited so non-reversible electrochemical effects in the electrode-tissue interface 
are minimized, and there is no permanent damage to the living tissue [2]. Because 
of inherent difficulties in performing in vivo experiments, modeling and simulation 
are valuable aids in designing retinal implants [6, 13, 15, 17]. 

The admittance method is attractive for bioelectromagnetic problems because it 
can solve complex heterogeneous models using a wide variety of electromagnetic 
stimulation types in a computationally efficient way. 



8.2 Quasistatic Numerical Methods: The Admittance Method 

Quasistatic electromagnetic methods have been successfully used over the last 
25 years to model bioelectromagnetic interactions; in particular, variations of the 
finite difference method - the admittance method - and its complement, the imped- 
ance method have proven useful for diverse bioelectromagnetic problems, including 
calculation of specific absorption rate (SAR) and novel ways to induce hyper- 
thermia in patients [1, 5]. Armitage and Ghandi were the first to study realistic 
models with generic lumped circuit element quasistatic electromagnetic numerical 
methods. Before them, other authors have used analytic formulations of gross 
geometric body tissues approximations such as cylinders and multilayered spheres 
(see references of [1]). Our research group extended the admittance method 
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formulation introducing a general two-dimensional multiresolution meshing 
algorithm, in which cells close to tissue boundaries are discretized with fine resolu- 
tion while cells surrounded by large areas of homogeneous space are progressively 
larger [3]; more recently, a three-dimensional method based on the same principle 
has also been developed [18]. 

In the impedance method, the electrical properties of the model are described 
using an impedance network; similarly, an admittance network is used in the admit- 
tance method; otherwise the methods are equivalent. For both methods there are 
several possible formulations suited to different electromagnetic stimulation mech- 
anisms, be it an external electromagnetic field, a capacitive electrode, a metallic 
electrode, an implanted coil, etc. For the rest of this chapter we will focus on the 
admittance method. 

The quasistatic constraint assumes that the highest significant frequency compo- 
nent of the electromagnetic fields involved in the simulation has a wavelength much 
larger than the size of the model simulated. Another way of looking at the validity 
of the quasistatic assumption is to consider if at the single highest frequency com- 
ponent used for excitation it is reasonable to assume that the phase differences across 
the model are negligible. For the model sizes used in this application, the quasistatic 
condition is met up to frequency components in the range of tens of megahertz. 

The general idea of the admittance method is to obtain an equivalent circuit 
model starting from the geometry of the implant and surrounding tissue and the bulk 
electrical properties of the substances involved, and then apply electrical or magnetic 
stimulation using ideal current and voltage sources. The resulting equivalent circuit 
is solved for the node voltages using circuit theory and a numerical linear solver. 
Branch currents can then be calculated from the node voltages and circuit compo- 
nents values. Noting that each node in the equivalent circuit will correspond to the 
location of a spatial point in the model, the numerical solution of the simulation 
using the admittance method is the current vector field and the matching scalar 
potential field at each significant point in the model. Electric field, current densities, 
equivalent impedances, etc., can then be derived from these results. 



8.2.1 Layered Retinal Model 

For retinal implants, detailed three-dimensional models of retinal tissues and 
surrounding areas can be constructed using a layered retinal model [17]. In the 
multi-layer retinal models each layer represents a different tissue type. The values 
for conductivity (a) used for the simulation results presented have been obtained 
from experimental measurements performed on a frog's retina [11]. The thickness 
values for each layer have been measured from electron microscopy images of a 
transverse cut of a mammalian retina [16]. The retinal layers considered and their 
thickness and conductivity are shown in Table 8.1. 

In addition to the retina, models for retinal stimulation often must include the 
implant, electrodes, current return, and surrounding tissue, including choroid 



Retinal Cell Excitation Modeling 



163 



Table 8.1 Properties of retinal layers used for layered 
retina model 



Retinal layer 


a (s m" 1 ) 


Thickness (u.m) 


Photoreceptors 


0.0198 


60 


Outer nuclear 


0.0166 


30 


Fiber/outer plexiform 


0.0143 


60 


Inner nuclear 


0.0153 


30 


Inner plexiform 


0.0555 


30 


Ganglion cell 


0.0143 


30 


Optic fiber 


0.0143 


30 



The conductivity values used for this set of simulations cor- 
respond to frog retinal tissue [11] 




M- 10 





X position (voxels) 



Fig. 8.3 (a) Simulation model is obtained from a geometric description of the retinal layered 
substrate, electrode array and current return. The 2.5 mm x 2.5 mm x 0.48 mm model was dis- 
cretized to a resolution of 10u,m, resulting in a mesh having 250x250x48 voxels. In this view, 
current sources are noted as triangles under the electrodes, and the position of the current return 
marked by the central triangle, (b) Transverse slice of the model at 45Y= 125, showing detail of 
the retinal layers and the implanted electrode array, dielectric backing and current return (topmost 
layer of assembly) 



(a = 0.92sm _1 ), sclera (rj = 0.50sm _1 ) and vitreous humor (o=1.5sm _1 ) [4]. The 
model used is depicted in Fig. 8.3. For the example presented in this chapter, a 25 
electrode array having cylindrical electrodes of 200 urn diameter spaced 500 urn 
between centers, and with the current return exposed to the vitreous humor on the 
back of the electrode array dielectric substrate has been included. Each electrode is 
constantly injecting - 50 uA for cathodic stimulation, and the simulation considers 
the model to be static, i.e. purely resistive. 

A discrete model is then obtained by spatially sampling a three-dimensional 
geometric description of the tissue and implant at regular intervals on the three 
Cartesian coordinate axes. This sampled model can then be interpreted as a collec- 
tion of voxels, each surrounding a sampled point and considered made of a single 
material. Since the thickness of individual retinal layers are in the order of tens of 
micrometers, to calculate electrical activity inside a layer the model must be 
resolved with voxels of size 15 urn or less. 

In order to reduce the voxel count and thus the size of the resulting linear system, 
the spatial sampling can optionally be made to create an expanding grid in one of 
more spatial dimensions and additional multiresolution techniques can be applied 
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to the discrete model. For purposes of simplifying the discussion we will consider 
a discrete model having uniform voxel sizes. 



8.2.2 Equivalent Electric Circuit 

An equivalent electrical circuit is then constructed from the discrete model, by placing 
circuit nodes at the vertices of each voxel, and calculating lumped resistors from 
the dimensions and conductivity of the material or tissue the voxel is made of 
(Fig. 8.4). Each voxel is considered subdivided in 12 sub-volumes, four along each 
of the coordinate axes, and the electrical resistivity for each sub- volume is used to 
calculate the equivalent lumped resistor value, as shown in (8.1), where W, H, and 
L are the width, height, and length of the sub-volumes being considered. For pur- 
poses of building the equivalent circuit, each equivalent resistor is placed at the 
edge between the accessible node for the sub-volume it models. The process is 
illustrated in Fig. 8.3. 




Fig. 8.4 Calculation of equivalent circuit for individual voxel, (a) Circuit nodes are places at each 
vertex of the voxel, (b) The voxel is considered subdivided in four sub-volumes along the X coor- 
dinate axis, (c) An equivalent resistor is used to model each of the sub-volumes in that direction, 
and placed in the edge in between the accessible nodes for that sub-volume. Equivalent resistors 
are calculated using (8.1). (d) The same process is then repeated for the Y and Z coordinates axis, 
resulting in the final circuit equivalent model for the voxel. After all voxels are processed, the 
circuit for the model is formed considering each shared vertex as a shared node. Finally, the current 
sources for excitation and the ground are placed 
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« = — ■ -• (8.1) 

W-H a 

An equivalent electrical circuit is created by considering all the resistors for every 
voxel, and knowing that contiguous voxels share two nodes at each shared edge. The 
ground node is assigned, and the stimulation is added using current sources. In the 
example case considered in this chapter we have used one current source per elec- 
trode; since we considered cathodic stimulation, for each current source the negative 
terminal is connected to a node belonging to an electrode and the positive terminal 
to ground. Additional circuit elements can also be added to the equivalent circuit to 
model other electrical behaviors of the tissue or implant. 



8.2.3 Electric Potentials and Current Density Magnitude 
Calculation 

This equivalent electric circuit can be solved using standard circuit theory tools. 
Using Kirchoff current law, the model's equations can then expressed as a linear 
combination, in terms of an admittance matrix G, an unknown voltage vector V, and 
a current vector I, as shown in (8.2). 

GV = I. (8.2) 

On a circuit having n nodes including ground, G is a symmetric, sparse, 
(n-1) x (n-1) admittance matrix, V is the unknown voltage vector, and I is the vector 
of independent current sources modeling the stimulating electrodes. While any 
method can be used for small systems, the size of the matrix G grows with the 
square of the number of nodes. Because of this, direct methods such as Gaussian 
elimination may not be the most suitable for this application; instead, iterative 
approaches including Krylov sub-space methods present advantages. In particular, 
the biconjugate gradient method with appropriate preconditioning has proven to be 
efficient in the considered cases (Figs. 8.5 and 8.6). 

The solution of the system provides the value for the potentials at each node of 
the circuit, which are equivalent to the potentials at the vertexes of each voxel. The 
components of the current density vector field at each point in the model can then 
be determined from the cross-sectional areas of each voxel, the equivalent resistors, 
and the branch currents flowing through each of them: 

/ x = y_L (8-3) 

For each voxel, the branch currents parallel to one coordinate axis are calculated 
by considering the voltage at the nodes and the four resistors along that direction. 
For the X axis, for example, the current density component is determined by taking 
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Fig. 8.5 Top: Model slice ( Y = 1 25) and contour plot of the electric potential resulting from admittance 
method simulation of the retinal prosthesis applying (center) cathodic and (bottom) anodic stimulation 
through the central electrode of the array. Units on the X and Y axes are voxels 
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Fig. 8.6 Contour plot of the electric potential resulting from admittance method simulation of the 
retinal prosthesis applying (a) cathodic and (b) anodic stimulation through all 25 electrodes. 
Values correspond to Y= 125 slice. Units on the X and Y axes are voxels 




into account the contribution of each sub-voxel in the X direction (8.3), where J x is 
the current density component for direction X, I is the branch current going 
through each of the resistors parallel to the X axis, and Y • Z is the transverse area 
for each sub-voxel in the X direction. The current densities in the remaining axes 
are calculated in a similar way. 

Figures 8.7-8.9 show the resulting current density magnitudes for the model 
slice considered in Fig. 8.5, and the values on a line at the ganglion cell layer, under 
the middle row of electrodes, for both the case of a single electrode and all 25 
electrodes excited. 



Retinal Cell Excitation Modeling 



167 



a 


1 electrode firing, current Density Magnitude [log(A/m )] 


40 




20 
10 


| | 



I 



50 



100 



150 



200 



250 



10 



-10 



b 


25 electrode firing, current Density Magnitude [tog(A/m )] 


40 




20. 
10 


^m ^B *i ^H 



i 



50 



100 



150 



200 



250 



Fig. 8.7 Colorplot of current density magnitude for (a) single stimulating electrode and (b) all 
electrodes stimulating simultaneously. The values plotted are from a transverse slice of the three- 
dimensional model cutting through the center of the central electrode, at the plane Y = 1 25 
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Fig. 8.8 Lineplot of current density magnitude at the GCL, centered under the central electrode, 
with a single electrode stimulating 

8.3 Three-Dimensional Activation Function Calculation 



In his early work, when analyzing a discretized formulation of Hodgkin and Huxley 
equations [7], and assuming initial conditions of transmembrane potential at rest, 
Rattay noted that the only dependency of the cellular transmembrane potential with 
respect to the extracellular potential V E was given by the relation described in (8.4), 
and called it the activation function [13,15]: 
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Fig. 8.9 Lineplot of current density magnitude at the GCL, centered under the central electrode, 
with all 25 electrodes stimulating 



1 dx 2 



(8.4) 



In (8.4), dx is the length differential along the axon of the neuron we seek to 
stimulate, taken in the direction of action potential propagation. Taking into 
account that the intracellular field of sensory neurons is very weak compared 
to the fields created by an implanted stimulator, in a retinal prosthesis the activa- 
tion function can be approximated as an exclusive function of V E . If calculated 
close to the cellular wall, the activation function is positive for zones that tend to be 
depolarized by the influence of the extracellular potential and negative for zones 
that are hyperpolarized, allowing to characterize a stimulation configuration in 
terms of its potential capability to trigger or not trigger action potentials in neural 
cells starting from the extracellular fields it creates (Fig. 8.10). 

When using layered retinal three-dimensional models and numerical methods, 
the activation function can be calculated if the scalar electric potentials are known 
for points along the axons. Note that the activation function describes a necessary 
but not sufficient condition for neural stimulation. 

In the case of the GCL, the axons we look to stimulate are assumed perpendicu- 
lar to the surface of the retina, so the activation function has been calculated along 
that direction. Note that while in general cathodic stimulation is desirable to anodic 
stimulation [14], in this particular case, because we assume that activation is taking 
place at the small portion of axon coming from the ganglion cell towards the elec- 
trode, anodic stimulation takes place. Had we positioned the stimulator array so 
excitation will happen in a segment of axon going away from the electrode instead 
of coming towards it, or had we considered that activation happened in a segment 
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Activation Function at GCL for 25 electrode firing ( max = 6.9e+007) 
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Fig. 8.10 Plot of normalized activation function corresponding to GCL from slice Y = 125. In this 
case, all 25 electrodes of the array are injecting current. The peaks in the figure align under a row 
of electrodes. The five central peaks correspond to one electrode each, and the lateral peaks cor- 
respond to the place where the dielectric backing of the electrode ends, allowing current flow to 
go up towards the vitreous and the current return at the back of the implant assembly. Traces for 
the anodic and cathodic stimulation are shown. From this plot we can conclude that for this particu- 
lar stimulator configuration, since the activation function is greater than zero under the electrodes 
in the anodic case, stimulation happens during the anodic phase of the pulse. In addition, during 
the cathodic phase the side peaks have a magnitude comparable to the electrode peaks for the 
anodic case - this is a reason of concern, hinting that perhaps a wider dielectric should be used 
for this array, to allow currents to spread through a larger volume and obtain a lower value for the 
activation function at that place 

of axon going away from the stimulating electrode, cathodic stimulation would 
have taken place instead. 

One way to think about why anodic stimulation takes place in this configuration 
is to consider that in anodic stimulation the electric potential at the stimulating elec- 
trode is positive with respect to all other points in the model; that implies that for an 
axon having its action potential propagation direction coming towards the stimulating 
electrode, the potential will be lower at points farther away from the stimulating elec- 
trode. The first and second derivatives of the voltage along the axon taken in the 
direction of action potential propagation will then be positive, making the activation 
function positive (Fig. 8.11). 



8.4 Safety of Implant 



The standards concerning electrical exposure safety [9, 10] describe different types 
of electromagnetic interactions with living tissue and report thresholds beyond 
which the body will react to the stimulus. Since the intention of the implant is to 
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Fig. 8.11 Plot of normalized activation function corresponding to GCL from slice Y= 125, for 
the case of only the central electrode injecting current. As the previous case, the activation func- 
tion takes positive values during the anodic phase of the pulse, indicating possible stimulation. 
Note that during the cathodic phase the side peaks are small compared with the side peaks in the 
simulation firing all 25 electrodes 



reach those threshold values, the safety standards must be interpreted in this case 
as a general guideline of the order of reasonable values for stimulation and not as 
a set limit for exposure. 

With reference to low-frequency electrical stimulation, the coupling mechanisms 
include conduction electric current, polarization of bond charges, and reorientation 
of existing dipoles [10]. In the case of retinal implants, because of the very low 
stimulation frequencies, conduction currents will be more noticeable than displace- 
ment currents. 



8.5 Conclusion 



Bioelectromagnetic modeling applied to simulating retinal implants is a complex 
and multidisciplinary topic. Variations of the admittance and impedance methods 
are suitable to solve this type of problems, but can only provide results as good as 
the underlying model. Understanding if a particular implant and excitation configu- 
ration will be successful at triggering action potentials in the target neurons requires 
taking into account the excitation patterns, surrounding tissue anatomy, and the 
mechanics of triggering action potentials in the target neural cells. 
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Part of what makes solving these systems very challenging is the fact that the 
size of the model tends to be large in comparison with the minimum feature size, 
resulting in extremely large linear systems, which can only be solved by using itera- 
tive algebraic methods. In addition, the material properties and consequently the 
current and voltage magnitudes involved can vary multiple orders of magnitude, 
making the convergence of the system harder for iterative solvers. Some of the 
models we are currently working with involve matrices having over 50 million rows 
and columns. These systems are being solved using multi-resolution techniques and 
sparse iterative linear solvers [17, 18]. 

The configurations analyzed in this article considered an intraocular current 
return. Characterizing performance for different current return configurations in 
epiretinal implants is complex. Part of the issue is that biological tissue in the area 
is arranged in layers, and the range of conductivities involved varies by several 
orders of magnitude. Further, if the current return electrode is implanted extraocu- 
larly and the eye retain movement after surgery, current densities will vary with 
the position of the eye as well. In general, as the electrode array is pressed into the 
retina, the current injected tend to penetrate the retinal surface under each of 
the active electrodes regardless of the current return configurations. The currents 
injected for each electrode will then seek a path towards the current return, and any 
asymmetry in the conductive path from the electrode through the tissue to the current 
return will result in different current density patterns; areas having shadows of 
lower current densities will appear. This shadow effect is more pronounced with 
larger and denser electrode arrays, and it is hard to characterize. Some of the current 
return related factors that affect performance include current return shape, size, 
material, surrounding tissue structures, and distance. 
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Chapter 9 

Neurotransmitter Stimulation for Retinal 

Prosthesis: The Artificial Synapse Chip 

Raymond Iezzi and Paul G. Finlayson 



Abstract Retinal prostheses may one day improve the lives of hundreds of thousands of 
patients with retinitis pigmentosa (RP) or millions of blind patients with advanced 
age-related macular degeneration (ARMD), depending on their effectiveness. While 
considerable progress has been made in electrical stimulation of the retina, herein 
we explore some possible alternatives to electrical stimulation for retinal prosthesis. 
Since neurotransmitters normally shape visual responses, some groups have been 
developing visual prostheses based upon the spatially and temporally controlled 
delivery of neurotransmitters to the retina. This chapter examines the possibilities 
for utilizing these chemical messengers, as a means to effectively stimulate retinal 
ganglion cells and produce vision along established visual information channels. 



Abbreviations 

5HT 5-Hydoxytryphan, serotonin 

AGB 1 - Amino-4-guanidobutane 

AMPA a-Amino-3-hydroxyl-5-methyl-4-isoxazole-propionate 

EAAT Excitatory amino acid transporters 

GABA Gamma-aminobutyrate 

iGluR Iono tropic glutamate receptor (GluRl, GluR2, GluR3, GluR4) 

INL Inner nuclear layer 

IPL Inner plexiform layer 

mGlur Metabotropic glutamate receptor 

NMDA /V-methyl-D-aspartate 

OPL Outer plexiform layer 

P# Postnatal day 

PR Photoreceptors 
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RCS Royal College of Surgeons 

RD1 Retinal degeneration type 1 mouse 

RGC Retinal ganglion cell 

RP Retinitis pigmentosa 

S334ter Opsin gene bearing a termination codon at residue 334 



9.1 Pathophysiology of Retinal Degeneration 

Two major classes of retinal disorders, retinitis pigmentosa and age-related macular 
degeneration, result in the loss of vision, due to progressive loss of photoreceptors 
(PR). Retinitis pigmentosa is a term used to designate diverse genetic disorders 
[15, 34, 38, 64] that vary in their hereditary linkage - autosomal recessive, auto- 
somal dominant, sex linked, mitochondrial or digenic, and in the underlying genetic 
mutations (see Chap. 3). Although, the onset, rate and type of PR loss vary between 
these genetic deficits, they all result in a progressive loss of photoreceptors. It also 
appears that several different factors, including genetic mutations play a role in PR 
degeneration in ARMD [14, 33, 56, 67] (see also Chap. 3). The diverse etiologies 
of RP and ARMD suggest that a single treatment will likely not be possible. Animal 
models of retinitis pigmentosa indicate that although further neurodegeneration and 
reorganization in the remaining neural retina occurs (see below), much of the rich 
network within the retina remains intact for extended periods of time. This presents 
the opportunity to produce visual sensations through the artificial stimulation of the 
degenerated retina. 



9.2 Modes of Interneuronal Communication 
Within the Normal Retina 

Although, excitatory (glutamate) and inhibitory (GABA and glycine) amino acids 
are the major neurotransmitter systems in the retina, other transmitters, including 
acetylcholine, serotonin, dopamine and a variety of neuropeptides shape the 
visual response (Table 9.1). There is a large diversity of receptors on retinal cell 
somata and dendrites in the inner and outer plexiform layers (IPL and OPL) and 
retinal ganglion cell layer (Table 9.2). The outer and inner plexiform layers are 
near enough to the subretinal and epiretinal surfaces, respectively, for effective 
activation by application of exogenous agents. In addition, the diversity and loca- 
tion of receptors may allow for differential stimulation of pathways, such as OFF 
and ON. 



9 Neurotransmitter Stimulation for Retinal Prosthesis: The Artificial Synapse Chip 175 

Table 9.1 Major transmitters released by retinal cells 



Cell type 


Transmitters 


Photoreceptors 


Glutamate 


Horizontal 


GABA 


Bipolar cells 


Glutamate 


Amacrine 




All 


GABA, glycine 


A4 


Glycine 


A8 


Glycine 


A10 


GABA 


A17, A18, A20 


GABA, serotonin 


A18 


GABA, dopamine 


A22 


GABA, substance P 


Starburst 


GABA, acetylcholine 


Retinal ganglion cells (RGC) 


Glutamate 



9.2.1 Outer Plexiform Layer 

In the OPL, glutamate release from photoreceptors directly modulates horizontal 
and bipolar cell responses. Horizontal cells exhibit AMPA type receptors, depolar- 
ize to glutamate release from PR, and reciprocally modulate photoreceptor 
responses through GABAergic neurotransmission. Horizontal cell transmission 
plays a major role in the surround inhibition, but not by GABAergic inhibition [36, 
65, 104]. GABA released from horizontal cell has a depolarizing effect on photo- 
receptors via GABA A receptors [54, 55], and modulates the temporal properties of 
light responses [43, 110]. Bipolar cell responses are dependent on the glutamate 
receptor types they express: OFF-center bipolar (human: flat midget bipolar 
(FMB)) express ionotropic glutamate receptors (AMPA) and hyperpolarize to light 
(like photoreceptors); whereas rod and ON-center cone bipolar cells (invaginating 
midget bipolar [1MB]), depolarize in response to light, due to decreased activation 
of g-protein coupled metabotropic glutamate, mGluR6 receptors [70, 72, 94, 95], 
which invert the signal from photoreceptors. Therefore, glutamate in the OPL 
differentially activates ON and OFF pathways. 



9.2.2 Inner Plexiform Layer 

Transmitter and interneuronal signaling in the IPL is more complex, but a simplifi- 
cation of the major interactions can be used to examine possible pathways for 
artificially stimulating the retina. 
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9.2.2.1 Bipolar Cell Excitation of Retinal Ganglion Cells 

Glutamate is released from cone bipolar cell ribbon synapses in the IPL, where it 
directly activates retinal ganglion cells (RGCs), through AMPA, NMDA and kain- 
ate receptors [10, 12, 42, 58, 59, 109], and also excites amacrine cells [3]. Rod 
bipolar cells do not directly contact RGCs, but contact amacrine cells (review: 
[2, 93]). Amacrine (All) cells contact cone bipolar cells and other All cells through 
gap junctions. Therefore, depolarization of All amacrine cells by ionotropic 
(AMPA and kainate) glutamate receptors leads to depolarization of cone bipolar 
cells and excitation of RGCs. 

9.2.2.2 Amacrine Cell Modulation of Signal Processing 

The many different types of amacrine cells in the IPL perform different functional 
roles, including lateral inhibition, and contributing to spatial tuning, direction selec- 
tivity and center surround receptive fields [13, 24, 88, 101]. For example, direction 
selectivity of RGCs involve GABA, likely originating in starburst amacrine cells 
[48]. Amacrine cells modulate bipolar, ganglion and other amacrine cells by releasing 
glycine, GABA, biogenic amines (5HT, dopamine and acetylcholine) and neuro- 
peptides in the IPL. In addition to ionotropic glutamate excitation, a subset of 
amacrine cells are likely modulated by metabotropic (mGlurl, mGlur2/3, mGlur5) 
receptors [9, 39, 49]. 

9.2.2.3 Inhibitory Transmitters 

Inhibitory transmitters differentially affect ON, OFF and rod pathways, based on 
cell type, pre- or post-synaptic action and current duration of GABA A , GABA C and 
glycine receptors [19-21, 68, 106]. GABA B receptor subunits Rla are found in the 
INL and RGC layer, while Rib is only found in the RGC layer [111, 112]. RGCs 
exhibit spontaneous and light evoked GABA and glycine responses [102], and are 
inhibited by GABA and glycine released by amacrine cells [24, 57]. In addition, 
OFF-bipolar cells are predominantly inhibited by glycine. Rod bipolar cells also 
receive glycinergic inputs but their major inhibitory response is through GABA 
acting on GABAj, receptors, which have slow kinetics, while ON bipolar cells are 
inhibited through GABA A receptors. 

9.2.2.4 Acetylcholine and Dopamine 

Acetylcholine is also released from amacrine cells [40]. Both muscarinic [32, 33, 
110] and nicotinic cholinergic receptors are found in retina (chick IPL and RGC 
layer [46]. Nicotinic alpha 7 receptors are also localized on bipolar, amacrine and 
ganglion cells in rabbit retina [17]. Nicotinic receptors are also expressed by a 
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subset of the starburst cells [47]. Acetylcholine (Ach) excites RGCs, particularly 
Y cells [18, 44, 46, 53, 63, 91], and a role for Ach has been implicated in direction- 
sensitivity in the retina. 

Dopamine released by amacrine cells regulates the spread of activity through 
gap junctions in the retina. Dopamine Dl receptors decrease the conductance of 
gap junctions between amacrine cells and bipolar cells [35, 107]. Therefore connec- 
tions are dynamically regulated in photopic and scotopic light conditions by dop- 
amine [4, 5]. 



9.2.2.5 Neuropeptides 

Amacrine cells also produce a number of neuropeptides, including substance P, 
somatostatin, vasoactive intestinal peptide (VIP), neuropeptide Y (NPY), corti- 
cotropin releasing factor (CRF) and opiates. The roles of peptides in retinal processing 
are less well understood, and due to the long-term instability of proteins, and 
complications in exogenous application of peptides, they are not likely to be useful 
in a neurotransmitter-based prosthesis. 



9.2.2.6 Putative neurotransmitters for retinal prosthesis 

The neurotransmitter and gap-junction interactions in the IPL and ganglion cell layer 
(GCL) provide a variety of means to stimulate the retina, possibly in a more natu- 
ralistic way. Glutamate application to the retina can directly excite RGCs, and indi- 
rectly activate RGCs through the amacrine-bipolar-RGC pathway. In addition, 
glutamate stimulation may activate amacrine pathways which are used for feature 
detection. Activation of amacrine cells can modulate many retinal processing path- 
ways. Acetylcholine may also be effective in selectively activating large ganglion cells 
such as the Y or type A RGC. In addition, GABA or glycine application could reduce 
activity and may also evoke rebound activity at the offset of application [30, 96]. 



9.3 Neurophysiological Changes in Retinal Degeneration 

An important consideration for any retinal prosthesis is how retinal function is affected 
beyond photoreceptor loss due to neurodegeneration and reorganization. Degenerative 
changes in biophysical and morphological cell properties, reorganization of connec- 
tions, endogenous transmitter release, and transmitter receptor alterations have been 
observed in animal models of retinitis pigmentosa [60-62]. Such changes may affect 
the excitability of RGCs to exogenously applied neurotransmitters. Late stages of retinal 
degeneration have been shown to severely limit RGC stimulation via electrical charge, 
as thresholds for eliciting electrically evoked cortical potentials increase and will likely 
impact the efficacy of neurotransmitter stimulation [45, 76]. 

Various animal models of RP express many similarities, but differ in time course 
of degenerative and physiological changes. Photoreceptor loss in the pink-eyed RCS 



9 Neurotransmitter Stimulation for Retinal Prosthesis: The Artificial Synapse Chip 179 

rat (rdy+/rdy+) is apparent by postnatal day 20 (P20), progresses rapidly to only a few 
nuclear layers by P40, and is nearly complete by PI 00 [6, 50, 69]. 
The S344-ter rat has a true rhodopsin gene mutation and therefore is an important 
model for studying human RP. Different lines of S344-ter rats exhibit different rates 
of progressive photoreceptor loss. In the rdl mouse model, which has a mutation in 
phosphodiesterase [7], PRs exhibit a rapid loss of in the first 2-3 postnatal weeks [22]. 
This early loss of PR is associated with abnormal development of bipolar mGLUR6 
receptors, and an early remodeling both in horizontal cells, which exhibit atrophy of 
terminal dendrites, and in rod bipolar cells, where photoreceptor directed dendrites do 
not develop [99]. However, amacrine cells do not appear to be affected [100], and in 
recent work from the same group the many types of RGCs also exhibit normal mor- 
phology in rdlO mice [99]. Bipolar and other cell remodeling occurs in stage 3, with 
onset varying with molecular deficit. In the RCS, s334ter, and P23H rat models, 
remodeling is relatively late in the disease with the onset on or after P270 [60, 62]. 

Visual function in RCS rats based on electroretinogram (ERG) recordings [8, 28, 
75, 79, 90] shows a progressive loss of rod function to near total loss by P100. Cone 
function, although declining, can be measured up to P200 [85]. Visual receptive 
fields in pigmented RCS rats are recorded in the superior colliculus up to PI 80, 
albeit with expected increases in threshold [90]. Thus, even after substantial loss of 
visual function due to photoreceptor loss, RGCs are relaying information to the 
central nervous system. 

Studies on degenerated retinas have in part focused on the changes in neurotrans- 
mitter levels and glutamate receptors. Glutamate and aspartate are reduced by 
approximately 50% in RCS rats at 23 weeks of age [77], and this is likely to be a 
consequence of photoreceptor loss. GAB A is reduced to a lower extent, while glycine 
levels increase in 23 week RCS rats [77]. However, other studies found that both 
GABA and glycine levels increase in degenerating retinas [23, 78, 92]. In addition, 
of the transmitters used by amacrine cells, dopamine is reduced by approximately 
50%, but acetylcholine levels are not affected [77]. The reduced dopamine levels 
correspond with a loss of dopaminergic amacrine cells associated with retinal degen- 
eration [16, 23]. A reduction or loss of many subunits of NMDA receptors (NR1, 
NR2A-D) has been found in RCS rat by PI 20 [29]. However, decreased expression 
of NMDA NR1 subunits in IPL was also observed in congenic non-pigmented rats 
compared to brown Norway [29]. Kainate binding sites also decrease by PI 80 in the 
IPL and OPL of RCS rats [98]. Excitation of RGCs can be shown in response to 
activation of AMPA, kainate and NMDA receptors [10, 58, 59, 109]. AMPA recep- 
tors subunit mRNA for GluR2, GluR3 and GluR4 increase in degenerating retinas 
of rdl mice by P40, but the flop:flip ratio (the ratio of the two AMPA receptor splice 
variants which affect binding and currents evoked by glutamate) is unchanged [71]. 
The levels of GluRl mRNA do not change, but the flop: flip ratio of Glurl (flip 
responses have slower desensitization and a greater steady-state component) does 
not exhibit the normal increase between plO and p40 [71]. 

The activation of RGCs by exogenous glutamate may also be affected by excita- 
tion of bipolar and amacrine cells. Bipolar cells express either mGluR or kainite 
glutamate receptors. Kainate receptor expression in the IPL and OPL is high at 
early stages of development (P17) and decreases by postnatal 180 days in pink-eyed 
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RCS rats [98]. Messenger RNA for metabotropic glutamate receptors (mGlur6), 
which are likely expressed by ON bipolar cells, increase in the INL of pink-eyed 
RCS rats at the longest (P60 and P120) periods examined [1], suggesting an up- 
regulation of these signal-inverting receptors. Physiological studies of RGCs 
in vitro have found many changes between p20 and pi 00, although there are conflicting 
results. Extracellular recordings in whole mount RCS rat retinas [86] demonstrated 
an increase in spontaneous activity up to PI 00, which coupled with decreased 
responses to light, resulted in significantly lower signal-to-noise levels. These 
investigators noted a predominance of cells with "OFF" responses by P47, and a 
decrease in receptive field size by P36. Intracellular recordings of RGCs in dystro- 
phic RCS rat retinal slices, however, demonstrated a decrease in the number of cells 
with sustained responses. Action potentials could not be evoked in 62% of RGCs 
from 9 to 12- week old animals [11]. 

Recent studies on functional glutamate receptors in retinal degeneration [61] 
based on cellular uptake of organic cations (AGB) found significant and differential 
changes in retinal cell glutamate responses. In two models of retinal degeneration 
due to loss of photoreceptors, rodless/coneless mice (rd/rd cl) and rhodopsin 
knock-in mutation model mice (brboG), a severe loss of glutamate (kainate) sensi- 
tivity of bipolar cells was found in the late, stage 3 of degeneration and remodeling. 
Glutamate still activates amacrine and ganglion cells, although reduced in this late 
stage of degeneration. However, in small islands where apparently non-functional 
cones survive, bipolar cells exhibit ionotropic glutamate responses. A high number 
of bipolar cells activated by kainate suggest that rod bipolar cells begin to express 
iGluR, in comparison to normally expressing mGluR [61]. In addition, AGB uptake 
suggests that some amacrine and ganglion cells exhibit increased activity. In a 
single retinal sample from the posterior pole of a human male RP patient with 
90-100% rod PR loss and remodeled cone PRs, all inner retinal cell types exhibited 
a robust glutamatergic response [61]. 

Overall, evidence from numerous studies indicates that despite decreased number 
and possibly excitability, surviving RGCs in retinas undergoing photoreceptor 
degeneration can transmit information to the brain. Glutamate receptor changes 
may reduce the efficacy of exogenous glutamate application, but this needs to be 
examined experimentally at specific points during the degenerative process. 



9.4 Rationale for a Neurotransmitter-Based Retinal Prosthesis 

Transmitter application may be a more effective and naturalistic means of conveying 
visual information to the brain, than other methods such as electrical stimulation. 
The effect of stimulation will depend on the location of application. In addition all 
types of retinal prosthesis and vision restoration strategies must be designed to 
stimulate the retina according to the type and stage of retinal degeneration. 

Subretinal application of neurotransmitters, such as glutamate, in the normal 
retina would inhibit ON and activate OFF ganglion cells, but these physiological 
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effects would be superimposed upon the effects of continuous glutamate release 
from photoreceptors. In eyes with retinal degeneration, however, exogenous gluta- 
mate application could replace endogenous glutamate release lost due to PR cell 
death. Pulsatile glutamate application would activate OFF ganglion cells and inhibit 
ON cells. Following glutamate application, the disinhibition of ON bipolar cells, 
may elicit a rebound response. Therefore, differential stimulation of OFF and ON 
pathways could be achieved, but the signals would be reversed - i.e. OFF cells 
respond first during stimulation, and ON-cells respond at the offset. Continuous 
release of glutamate with reductions to mimic light responses could mimic normal 
photoreceptor releases. However, this would likely produce too high of a glutamate 
load on cellular systems which clear glutamate from the extracellular space, such 
as excitatory amino acid transmitter pumps (EAATs). The distance from the sub- 
retinal surface and OPL in normal retina is over 100 urn, whereas after PR loss this 
distance can be less than 50 urn. 

Neurotransmitter application at the epiretinal surface can stimulate the retina by 
activating receptors in the ganglion cell layer (40-60 |im from surface) and in the 
IPL (60-75 |im from the surface). Epiretinal glutamate or acetylcholine application 
could directly activate RGCs through receptors on their somata, which are within 
50|im of the surface. Glutamate could also stimulate receptors in synapses within 
the IPL, including RGC dendritic fields, bipolar-ON cell synapses in IPL b, bipolar- 
OFF cell in IPL a, rod bipolar-amacrine in IPL, and amacrine cells. Epiretinal 
application of GABA or glycine could be used to inhibit RGCs and amacrine cells. 
This may be useful if for example RGCs become highly active in degenerated retinas, 
as those observed in rdl mice [97]. Inhibition of amacrine cells could result in 
disinhibition of other cells, including RGCs in the surround area, due to decreased 
inhibitory transmitter release. Our preliminary results indicate that ganglion cells 
exhibit robust excitatory responses to exogenously applied glutamate in 180 day 
RCS and s334ter line 4 rats. We also have observed that spontaneous firing rates of 
RGCs in these animals range from absent to high in degenerating retinas. 



9.4.1 Limitations of Electrical Stimulation 

Prostheses based on electrical stimulation of the retina have been under develop- 
ment over the past two decades. Testing in acute humans studies have had limited 
success in providing useful vision. Chronic human experiments have been limited 
to low-resolution devices, since large electrodes are required to handle the high 
currents required to stimulate degenerated retinal tissues. Small-diameter elec- 
trodes, required for a high-resolution prosthesis, are prone to failure due to high 
charge-densities that erode metals and stimulation voltages that often exceed those 
required to dissociate water. These facts make small-diameter electrodes more 
capable of inducing retinal tissue damage from free radicals that are toxic to the 
lipid membranes of neurons and glia. Further limiting the efficacy of current stimu- 
lation methods is the fact that electricity cannot selectively stimulate specific types 
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of visual pathways (e.g. ON and OFF channels) within the visual system. Thus, at 
this point, electricity cannot encode important sensory features used in normal 
central visual processing. 



9.4.2 Requirements and Benefits of Neurotransmitter 
Stimulation 

Many of these limitations could be circumvented by using more naturalistic means 
of stimulating retinal ganglion cells (RGCs) for a retinal prosthesis. Natural vision 
is encoded as neurotransmitter signals. Neurotransmitter-based retinal prosthesis 
designs will enable us to design and build a device based upon the physiological 
requirements for RGC stimulation by exogenous neurotransmitters in retinal degen- 
eration. Our preliminary results show that glutamate is effective in stimulating retinal 
ganglion cells. RGC responses to exogenously applied glutamate are brief, since 
excitatory amino acid transmitter transporter systems (EAATs) rapidly remove 
glutamate from the extracellular space. A neurotransmitter-based retinal prosthesis 
could also take advantage of other transmitters, simultaneously. For example, OFF 
responses may be mimicked by applying inhibitory transmitters such as glycine or 
GABA. By applying these inhibitory neurotransmitters adjacent to areas of gluta- 
mate stimulation, we may be able to simulate visual contrast. This approach is not 
possible with electrical stimulation alone. Finally, effective prostheses may use 
both transmitter and electrical stimulation, synergistically. However, the parameters 
for stimulating RGCs using glutamate and other neurotransmitters in diseased retinas 
have not been established. 



9.5 Technical Considerations and Design Approaches 

9.5.1 Operating Principles for a Neurotransmitter-Based 
Retinal Prosthesis 

Retinal prosthetic devices produce artificial vision by replacing the function of 
photoreceptors lost due to retinal degeneration. Ideally, these devices pattern affer- 
ent stimulation to the remaining retinal neurons, in spatially and temporally natu- 
ralistic patterns. Electrically based retinal prosthetic devices initiate neuronal 
stimulation by inducing local electrical fields that activate voltage-gated ion chan- 
nels. Neurotransmitter-based devices are capable of directly stimulating or inhibit- 
ing neurons by selectively activating ligand-gated ion channels. This requires 
stimulation hardware capable of accurately modulating the localized delivery of 
neurotransmitters in space and time. While microelectronic circuitry for the control 
of release has evolved considerably over many decades, methods for delivering 
neurotransmitters with these devices are still within relatively early stages of 
development. 
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9.5.2 Establishing a Retinal Prosthesis/Synaptic Interface 

9.5.2.1 The Proximity Requirement 

Prior to the fabrication of microfluidic devices for retinal prosthesis, the general 
requirements for retinal stimulation via neurotransmitters must the considered. It 
should be noted that inter-neuronal communication occurs primarily at the synapse. 
Thus, neurotransmitter-based retinal prosthesis devices must localize their delivery 
to retinal layers that contain synapses for the target cells of interest. Proximity 
between target dendrites and sites of neurotransmitter delivery is critical for two 
primary reasons. First, diffusion is a relatively slow process that will increase the 
latency between stimulation and response, significantly reducing the effective 
stimulus update rate. Taking into account the tissue tortuosity factor, the coefficient 
for diffusion of L-glutamate, the primary excitatory retinal neurotransmitter, at 
37°C is approximately 10 x 10" 6 cm 2 s -1 [37, 74]. This translates to a linear diffusion 
rate of approximately 33 |im/s. Thus, if the site of neurotransmitter release is 33 urn 
away from the target dendrites, the response latency will be 1 s. Limited to diffu- 
sional delivery, neurotransmitter-based retinal prostheses would be constrained to 
very low frame rates. Proximity is also critical for efficient delivery of neurotrans- 
mitter to target synapses. The concentrations required to elicit neuronal responses 
to the exogenous application of L-glutamate are relatively high (see discussion 
below). Thus, diffusional dilution over longer distances would necessitate higher 
total doses of L-glutamate. In addition, excitatory amino acid transmitter pumps 
actively remove L-glutamate from the extracellular space. This is desirable in that 
these pumps rapidly dampen neuronal responses to the exogenous application 
of L-glutamate, improving the dynamic range, spatial and temporal resolution of 
response. However, if there is poor proximity between stimulation sites and target 
dendrite populations, these pumps may increase the threshold quantity of L-glutamate 
release required to achieve neuronal stimulation. 

The proximity requirement for neurotransmitter-based retinal prostheses may 
necessitate that these devices penetrate into dendritic retinal sublaminae of the 
inner or outer plexiform layers. The concept of chemically inducing neurons to 
extend synaptic contacts to a retinal prosthesis has been proposed [51, 52, 66]. 
Epiretinal or sub-retinal neurotransmitter-based retinal prostheses or versions of 
these devices that penetrate into the retina could, incorporate drug-delivery methods 
to release chemo-attractant molecules that induce the migration of dendrites toward 
stimulation sites. The loss of afferent input to bipolar cells due to photoreceptor cell 
loss in retinal degeneration does induce bipolar cells to re-direct their dendrites 
toward the inner retina where they have been reported to create self-stimulation 
loop circuits [60-62]. Thus, there may be a period of time during which these de- 
afferented bipolar cells may be induced to synapse upon a sensory substitution 
implant. This may occur as a consequence of the sensory substitution, itself. Or, 
perhaps the controlled release of growth factors from a retinal prosthetic device 
could provide a signal to dendrites that would promote the extension and mainte- 
nance of synapses to the device. Retinal ganglion cells maintain their synaptic 



184 R. Iezzi and P.G. Finlayson 

contacts with their afferent bipolar and amacrine cells and do not become 
de-afferented as a consequence of the retinal degeneration. Thus, it may be more 
difficult to induce these cells to alter their well established dendritic organization. 



9.5.2.2 Convective Delivery of Neurotransmitters Via Microfluidics 

To overcome the temporal constraints of neurotransmitter diffusion some retinal 
prosthesis designs employ microfluidic technology capable of convective delivery. 
Two groups have worked on the development of microfluidic devices, capable of 
the controlling the release of neurotransmitter in space and time. Iezzi and col- 
leagues at Wayne State University first introduced the concept of a microfluidic 
neurotransmitter-based retinal prosthetic device [41]. Devoid of valves, the design 
employs the use of phototriggered neurotransmitters. These neurotransmitters do 
not activate ligand-gated ion channels prior to their flash photolysis. The "uncage 
and release" device employs microfluidic channels that incorporate an optical sub- 
system for the spatially and temporally controlled activation of phototriggered 
neurotransmitters. An electrical current is then used to iontophoretically and/or 
electro-osmotically eject the charged, uncaged neurotransmitter from a microflu- 
idic aperture or microneedle into close proximity to the target dendrites. This 
design involves storing a reservoir of caged L-glutamate prodrug and involves optical 
and electrical means for controlled release. This potentially minimizes the possibility 
of a dose-related L-glutamate induced excitoxicity. Finlayson and Iezzi [80] have 
shown that the localized convective delivery of L-glutamate via pneumatic ejection 
results in linear RGC dose-response firing with response latencies of 200ms. These 
preliminary results validate the utility of convective neurotransmitter delivery for 
retinal prosthesis. 

Another group at Stanford University has also developed microfluidic circuits 
that employ electroosmotic flow for the controlled delivery of neurotransmitters in 
space and time. They have demonstrated that electric field-driven fluid ejection of 
bradykinin was effective in stimulating PC- 12 cells cultured on the stimulation 
system [81-84]. 



9.5.2.3 Functionalized Surfaces for Neurotransmitter Stimulation 

Pepperberg and associates have been developing functionalized surfaces coated with 
tethered neurotransmitters for neuronal stimulation [73, 89, 105, 108]. According to 
the design concept, an electrical or other control signal will modulate the capacity 
of tethered molecules to bind to synaptic or extra- synaptic neurotransmitter recep- 
tors. Neurotransmitter analogs such as the muscimol, bound to biotin for the future 
purpose of adsorption to surfaces, rendering them "functionalized" have been shown 
to activate GABA receptors in an oocyte model. Since the neurotransmitter-biotin 
conjugates will ultimately be adsorbed to the surface of the implant, solid posts 
could be used to assure that stimulation occurs within the desired retinal layers. 
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9.5.2.4 Synaptic Requirements for L-Glutamate Mediated 
Neuronal Stimulation 

Any system for delivering neurotransmitters to the retina for the purpose of retinal 
prosthesis will be required to match doses of L-glutamate required by target neu- 
rons. Consequently, an analysis of the anatomy and physiology of the synapse may 
be useful in establishing operating parameters for neurotransmitter-based retinal 
stimulators. 

The requirements for neurotransmitter stimulation of the retina differ according 
to the target cells for stimulation. ON and OFF pathways are first established at 
the bipolar cell level. Thus, stimulation at this level may permit selective ON and 
OFF stimulation selectivity. Depending upon whether the retinal prosthesis is 
placed epiretinal or subretinal, microneedles may be necessary to deliver neu- 
rotransmitter to target neuronal cell dendrites. In degenerating retina, bipolar 
cells that have lost their photoreceptor input redirect their afferent dendrites 
toward the inner plexiform layer (IPL). Within the IPL RGC afferents synapse. 
Neurotransmitter stimulation directed toward ganglion cells must reach this 
region. Within the IPL, it may be possible to stimulate bipolar cell dendrites and/ 
or RGCs directly. 

The rate of quantal excitation to RGCs in response to visual stimulation has 
been examined. Any neurotransmitter-based retinal prosthesis will need to 
mimic patterns of quantal excitation induced by visual stimulation. Freed deter- 
mined that the just-maximal sustained RGC response to visual stimulation was 
induced by 3,700 quanta of L-glutamate per second, among all synapses [25, 26]. 
Studies of the number of L-glutamate molecules per synaptic vesicle report a 
range between 500 and 10,000 [87]. Thus, between 1.85 and 37 x 10 6 L-glutamate 
molecules per second would be required to induce a sustained RCG response. 
Freed and Sterling reported that there are approximately 550 bipolar synapses 
upon an ON alpha-RGC in the area centralis [27]. At 10° eccentricity, the larger 
membrane surface area of ON alpha-RGCs causes them to have approximately 
2,200 bipolar cell synapses, since the density of bipolar cell synapses on the mem- 
brane is constant [25, 26]. Based upon a synapse diameter of 200 nm 2 and a 
synaptic cleft of 20 nm, the volume of each synapse is approximately 2.5 al 
[103]. Thus, the total synaptic volume for a single ON alpha-RGC ranges 
between 1.38 fl near the area centralis and 5.5 fl at 10° eccentricity. Using the 
lowest molar quantity of L-glutamate needed for sustained RGC stimulation, 
combined with the largest total synaptic volume for an ON alpha RGC we arrive 
at a predicted minimum molar concentration of L-glutamate necessary for stimu- 
lation by a neurotransmitter-based retinal prosthesis of 0.55 mM L-glutamate. By 
taking the higher molar quantity of L-glutamate from the above computations, 
divided by the smallest total synaptic volume for an ON alpha-RGC, we predict 
that the upper concentration for L-glutamate required for sustained stimulation 
is 11.1 mM. This range is consistent with our unpublished experimental findings 
for RGC stimulation via exogenous application of L-glutamate in normal 
Sprague-Dawley, RCS and S334-ter-4 rats. 



186 R. Iezzi and P.G. Finlayson 

9.6 Summary 

A neurotransmitter-based retina prosthesis is a feasible option for restoration of 
visual function in humans with retinal degeneration. The diverse and differential 
actions of glutamate, GABA, glycine and acetylcholine on surviving retinal cells 
allow for both excitatory and inhibitory stimulation of the retina. These neurotrans- 
mitters differentially influence a number cell types that underlie feature detection 
processed in the retina. Although, pathophysiological changes in retinal degenera- 
tion may reduce the effectiveness of stimulation, neurotransmitter based prostheses 
offer the ability to activate retinal circuits, or suppress hyperactive ones. Technical 
considerations such as diffusion and EAATs ensure that exogenous local applica- 
tion of transmitters will affect a small restricted area of retina with responses that 
are temporally dampened. In addition, since passing axons are not stimulated, a 
neurotransmitter-based retinal prosthesis can maintain a high spatial specificity, 
even at suprathreshold stimulation. Technological advances in electrical prosthesis 
will aid in the development of a neurotransmitter based prosthesis, since existing 
circuits may be used to control electro-osmotic ejection of transmitters. In addition, 
there are new advances in other drug delivery technologies, such as caging and 
tethering molecules, which may be adapted for a neurotransmitter based retinal 
prosthesis. In conclusion, neurotransmitters offer a promising new approach to 
stimulation for retinal prosthesis, which may also supplement and piggy-back upon 
existing technology. 
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Chapter 10 

Synthetic Chromophores and Neural 

Stimulation of the Visual System 

Elias Greenbaum and Barbara R. Evans 



Abstract This chapter presents an overview of optical stimulation of neural cells 
by synthetic chromophores and their potential use in the field of artificial sight. 
The chromophores and techniques that are discussed include azo chromophores, 
photo release of caged neurotransmitters, pore blockers and photoisomerization, 
the channelrhodopsins, melanopsin, and the Photosystem I reaction center of green 
plants. 



Abbreviations 

ATR All-trans retinal 

ChR Channel rhodopsin 

Cy5 Red-emitting cyanine-based fluorescent dye 

DIC Differential interference contrast microscopy 

FITC Fluorescein isothiocyanate 

PSI Photosystem I reaction center 

UV Ultraviolet light 



10.1 Introduction 

Rods and cones contain the light-absorbing chromophores of the retina that trigger 
the primary events of vision. The light absorbing molecule in the discs of rod cells is 
rhodopsin, comprised of opsin, a protein, and 1 1-cw-retinal, a Vitamin A derivative. 
As illustrated in Fig. 10.1, absorption of a visible photon triggers the isomerization of 
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Fig. 10.1 Vision begins by photon absorption in the chromophore 1 1-cis-retinal with is converted 
to the all trans isomer 



the \\-cis isomer to the all- trans isomer. This cis- trans isomerization activates a G 
protein cascade that sets in motion the molecular events in the rod outer segment that 
result in visual perception [31]. Visual diseases such as age-related macular degenera- 
tion or retinitis pigmentosa are characterized by loss of the first step in vision, the 
phototransduction cascade in the photoreceptor outer segment. Much of the remain- 
ing neural pathway from retina to brain remains intact. Stimulation of these surviving 
retinal neurons is the biomedical engineering basis of multielectrode retinal prosthetic 
devices [12]. However, considerations of geometry, stability and fabrication of elec- 
trodes plus power requirements and the physics of electric field propagation in con- 
ductive media place a practical upper limit on the number of electrodes. The use of 
synthetic chromophores for the optical stimulation of retinal neural cells presents an 
attractive, if challenging, alternative. Optical stimulation as a tool for studying neural 
systems is a well established idea. As noted by Zhang et al. ". . it will be a physiolo- 
gist's dream-come-true to simply sit back and let light beams stimulate and assay the 
operation of a well-defined excitable tissue, such as a neural circuit" [38]. Multiple 
approaches to optical stimulation of cells that are not normally light-sensitive are 
known. A logical extension of this work is the application of synthetic chromophores 
to in vivo stimulation of the visual system. This idea is the molecular analog of 
multielectrode prosthesis stimulation of neural cells and is the focus of this chapter. 
Optical stimulation of neural cells can be viewed in at least two ways: as a useful 
experimental technique to expand our knowledge of neuroscience and mapping of 
neural pathways, or as a biomedical engineering approach to the development of 
molecular prosthetic structures that might be capable of replacing multielectrode 
visual prosthetic arrays. The contemplated advantages of using molecular chro- 
mophores for optical stimulation of the retinal cells are their nanometer size, direct 
interaction with the neural membrane, and ability for spectral tuning. External power 
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sources, in principle, are not necessarily needed. The application of synthetic chromophores 
to artificial sight is the molecular analog and possible successor to multielectrode 
arrays for the stimulation of neural cells. Three broad approaches have been pro- 
posed: (1) chemical modification of ion channels, usually with derivatives of the 
photoisomerizable chromophore azobenzene [2, 32]; (2) photochemical release of 
signaling molecules [4, 20, 37]; and (3) application of light-sensitive proteins such as 
channelrhodopsin [3, 13, 19, 21] or Photosystem I reaction centers [10, 14]. 



10.2 Pioneering Experiments 

10.2.1 Stimulation with No Chromophores 

We begin by noting one optical stimulation technique that dispenses with synthetic 
chromophores entirely. Fork reported observation of light-induced neural activity 
when laser irradiation was applied to the abdominal ganglion of the marine mollusk 
Aplysia calif ornica [9]. Neural cells were impaled with conventional microelec- 
trodes and illuminated with a laser beam with a minimum spot size of 10 urn. Laser 
stimulation of the cells with blue (488 nm) or green (515nm) light produced firing 
with the light pulses "on" in some cases and "off in others. In other experiments, 
especially those with the addition of ouabain, firing occurred during the light pulses 
whereas in others, firing occurred when the laser beam was switched off. In this 
work, none of the cells was selected for photoreceptor activity. Fork concluded that 
the laser-induced signals were caused by a mechanism other than damage. However, 
detailed work by Hirase et al. [11] indicated that relatively low power laser irradiation 
can produce reactive oxygen species and higher powers can result in membrane 
damage. Nonetheless, a significant result of Fork's work is that intense local elec- 
tromagnetic disturbances can induce neural activity in cells. 

The intensity of the laser beams used for this prior work was high. For example, 
a typical response of a silent cell in normal seawater used a 12.5 mW beam at 
488 nm. This corresponds to an irradiance of 1.6xl0 8 W/m 2 . For the experiments 
using ouabain, a beam power of 4.5 mW was used, corresponding to an irradiance 
of 5.7 x 10 7 W/m 2 . The corresponding solar irradiance at noon on a clear day is of 
the order 1 x 10 3 W/m 2 . This early work pointed in the correct direction for optical 
stimulation of neural cells. The use of synthetic chromophores with tailored light 
absorbing properties can be expected to greatly reduce the intensity of light that is 
required to achieve a specific optoneural effect. 



10.2.2 Azo Chromophores 

One workhorse for optical modulation of neural cells is the azo class of synthetic 
chromophores that are characterized by the light-induced trans-cis isomerization 
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Fig. 10.2 (a) Azobenzene can be converted from the trans to the cis state photochemically, and 
will revert back to the stable trans state thermally. Alternately, the cis to trans conversion can be 
effectuated with a distinct wavelength of light, (b) Simplified state model for azobenzene chro- 
mophores. The extinction coefficients are denoted £, whereas the quantum yields for the photoi- 
somerizations are labeled 0. The rate of thermal relaxation is denoted by k. Competition between 
these pathways determines the composition of the photo-stationary state [from ref. [36], Elsevier 
© 2006, used with permission] 



(and its reversal) about the double nitrogen bond: the azo linkage, -N=N-. The 
parent molecule for the early studies of this chromophore is azobenzene, illustrated 
in Fig. 10.2, in which the azo linkage bridges two phenyl rings. Azobenzene and its 
myriad substituent derivatives are often referred to as "photoswitches." However, 
the term "switch" is not quite right as the word is commonly understood: a device 
with two stable states. Azobenzene does not have binary stability. Only the trans 
state is stable. The cis state is more energetic by 49kJmol _1 (in heptane) [7]. The 
rate of decay for substituted derivatives depends on the specific molecule. The 
lifetime of azobenzenes is on the order of hours, but is considerably less for amino- 
azobenzenes and pseudo-stilbenes [36]. Continuous irradiation of azobenzene 
produces mixtures of photostationary states of the trans and cis isomers whose rela- 
tive concentration is wavelength dependent. 350 nm light is preferentially absorbed 
by the trans isomer and populates the cis state, whereas 450 nm light accelerates 
conversion of the cis form back to the trans state [30]. 

Lester and Nerbonne provided a succinct summary of the way in which physi- 
ological systems can be manipulated by light: a physiological parameter is monitored 
while photochemistry is used to alter the physiology of the system being 
monitored [18]. Deal, Erlanger and Nachmansohn showed how carbamylcholine- 
produced depolarization of the excitable membrane of the monocellular electroplaques 
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preparation of Electrophorus can be regulated by light [6] . They worked with two 
photoisomerizable compounds: (1) ,/V-/?-phenylazophenyl-Af-phenylcarbamylcho- 
line chloride and (2) p-phenylazophenyltrimethylammonium chloride. The trans 
photostationary state of each predominates under 420 nm light, whereas the cis 
version is the majority species under 320 nm irradiation. Both isomers inhibit 
depolarization of the membrane. However, the trans isomer is a stronger inhibitor 
than the cis isomer. The result of Deal et al. is an early example of photoregulation 
of the potential difference across an excitable membrane by exposing electro- 
plaques to light of appropriate wavelengths in the presence of a solution of 
carbamylcholine and either of the two compounds. The work illustrated coupling 
a cis-trans isomerization, the first step in the initiation of a visual impulse, with 
substantial changes (20-30 mV) in the potential difference across an excitable 
membrane. 

Lester et al. [17] prepared a covalently bound photoisomerizable agonist and 
compared it with reversibly bound agonists at Electrophorus electroplaques. Light- 
flash experiments with tethered 3-(a-bromomethyl)-3'-[a-(trimethylammonium) 
methyl] azobenzene (QBr) resemble those with the reversible photoisomerizable 
agonist, 3,3',fow-[a-(trimethylammonium)mefhyl]-azobenzene (Bis-Q): the con- 
ductance is increased by cis — > trans photoisomerizations and decreased by 
trans — > cis photoisomerizations. As with Bis-Q, light-flash relaxations had the 
same rate constant as voltage-jump relaxations. Receptors with tethered c/s-QBr 
have a channel duration severalfold briefer than with the tethered trans isomer. By 
comparing the agonist-induced conductance with the cis/trans ratio, Lester et al. 
concluded that each channel's activation is determined by the configuration of a 
single tethered QBr molecule. 

Balasubramanian et al. embedded azobenzene and azobenzene-/?-carboxylic 
acid methyl ester in a model membrane system that was the microemulsion obtained 
by the dispersion of water and hexadecane using amphipathic potassium oleate as 
the emulsifier and hexanol as the cosurfactant [1]. This work demonstrated light- 
induced alteration of the electrical conductivity in the birefringent lamellar multibi- 
layer system that was attributed to the optical activity of the azo chromophores. 
This work also demonstrated light-induced alteration of the ester hydrolytic activity 
of a-chymotrypsin dissolved in the membrane containing azobenzene and azoben- 
zene ester separately. Other applications of azo chromophores to biological systems 
have been studied [16, 28, 29, 35]. 



10.3 Current Research 

10.3.1 Caged Neurotransmitters 

As illustrated in Fig. 10.3, when illuminated with UV light or by multiphoton excita- 
tion [33], caged amino acid neurotransmitters are converted into biologically active 
amino acids that can rapidly initiate neurotransmitter action. These caged probes 
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Fig. 10.3 U V light or multiphoton excitation of caged amino acid neurotransmitters can be converted 
into biologically active amino acids 

provide a means of controlling the release - both spatially and temporally - of agonists 
for kinetic studies of receptor binding or channel opening. The technique of rapid 
light-induced release of signaling molecules [34] descends from the flash photolysis 
experiments of Norrish and Porter [24]. Calloway and Katz pioneered a photochemical 
approach for high-spatial-resolution mapping of functional circuitry in living mam- 
malian brain slices [4]. Photostimulation was achieved by bathing brain slices in a 
molecularly caged form of the neurotransmitter glutamate [L-glutamic acid alpha- 
(4,5-dimethoxy-2-nitrobenzyl) ester], which was then converted to the active form by 
brief pulses (<lms) of ultraviolet irradiation. Using this technique, the locations of 
neurons making functional synaptic connections to a single neuron were revealed by 
photostimulation of highly restricted areas of the slice (50-100 urn in diameter) while 
maintaining a whole-cell recording of the neuron of interest. 



10.3.2 Pore Blocker and Photoisomerization 

Building on the work of Lester et al. [17], Banghart et al. [2] used structure-based 
design to develop a chemical gate that confers light sensitivity to an ion channel for 
remote control of neuronal firing using a pore blocker and photoisomerizable 
azobenzene structure. Figure 10.4 illustrates the basic idea. Bistable positioning of 
the pore blocker was achieved with light of two different wavelengths. Absorption 
of a 500 nm photon triggered a cis-trans isomerization. The -1.7 nm length of the 
trans isomer moved the blocker to the pore of the ion channel. Conversely, absorp- 
tion of a near UV 380nm photon triggered a trans-cis isomerization, The -1.0 nm 
length of the cis isomer caused retraction of the pore blocker. The light-activated 
gate was covalently linked to the ion channel and the ion channel was integral to 
the neuronal cell membrane. The control over individual neurons was spatially 
accurate and did not rely on diffusible ligands. Also, the gate could be reversibly 
photo switched, allowing recurrent control of neural activity. Inside-out patches 
from an oocyte were treated with lOOuM of the triad maleimide + azo linkage chro- 
mophores - triethanolamine for 30min. The patch showed a large Shaker current in 
380 nm light (cis) and almost complete block in 500 nm light (trans). Current block 
in the dark followed a biexponential time course with x 1 = 0.49 min and x, = 4.79 min. 
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Fig. 10.4 Application of azobenzene chromophores to the gating of ionic currents through modified 
Shaker channels. MAL is maleimide for cystine tethering. QA is a quaternary ammonium group to 
block the channel [from ref. [2], Nature Publishing Co., © 2004, used with permission] 



Volgraf et al. [32] applied the azobenzene technique to a ligand-gated ion channel, 
the ionotropic glutamate receptor (iGluR). Using structure-based design, they 
modified the ligand-binding domain to develop a light-activated channel. An ago- 
nist was covalently tethered to the protein through an azobenzene moiety, which 
functioned as the optical switch. The agonist was reversibly presented to the bind- 
ing site upon photoisomerization, initiating domain closure and channel gating. 
Photoswitching occurred on a millisecond timescale, with channel conductances 
that reflect the photostationary state of the azobenzene at a given wavelength. 



10.3.3 The Channelrhodopsins 



Nagel et al. have shown that Channelrhodopsins 1 and 2 (ChRl and ChR2) are 
involved in generation of photocurrents of the green alga Chlamydomonas reinhardtii. 
ChRl is a light-gated proton channel [22]. ChR2, on the other hand, is a directly light- 
switched cation-selective ion channel [23]. It opens rapidly after absorption of a photon 
to generate a large permeability for monovalent and divalent cations. Nagle et al. have 
demonstrated that ChR2 may be used to depolarize cells by illumination [23]. Boyden 
et al. [3] and Li et al. [19] achieved temporally precise, noninvasive control in 
well-defined neuronal populations by adapting the naturally occurring algal protein 
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Fig. 10.5 Neurons expressing yellow fluorescent protein-tagged Channelrhodopsin-2 and a voltage 
trace showing photo-stimulation-elicited spikes. From D. Evanko (2005), Nat Methods, 2: 
p. 726-7. Nature Pub. Co. © 2005. Used with permission 

Channelrhosopsin-2 (ChR2), a rapidly gated light-sensitive cation channel. The 
technique used lentiviral gene delivery in combination with high-speed optical 
switching to photostimulate mammalian neurons. The work demonstrated reliable, 
millisecond-timescale control of neuronal spiking, as well as control of excitatory and 
inhibitory synaptic transmission. Figure 10.5 illustrates neurons expressing yellow 
fluorescent protein-tagged Channelrhodopsin-2 and a voltage trace showing photo- 
stimulation-elicited spikes. The first 315 amino-acid residues of C. reinhardtii 
Channelrhodopsin-2 coupled to retinal can be used to impart fast photosensitivity [3, 
13, 19, 23]. ChR2 is a seven-transmembrane protein with a molecule of all-trans retinal 
(ATR) bound at the core as a photosensor [23]. Upon illumination with -470 nm blue 
light, ATR triggers a conformational change to open the channel pore. Since ChR2 is 
a light-sensitive ion channel, the expected fast response was indeed observed, within 
50|is of illumination [3]. Combining ChR2 with fast light switching made it possible 
to activate neurons with the temporal precision of single action potentials [3]. 



10.3.4 Melanopsin 



Another route that has been examined is the increase and relocation of intrinsic 
mammalian visual receptors through transgenic ecotopic expression to restore 
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photosensitivity to the retina. Melanopsin is a retinal-containing, photosensitive 
protein which is expressed at low levels in neuronal cells, including a small subset 
of the retinal ganglia. It does not directly act to generate a membrane potential, but 
transfers the visual stimulus through a signaling pathway. Utilization of melanopsin 
would have the advantages of using an intrinsic mammalian retinal protein with a 
visual pigment similar to that of the natural photoreceptors, but has the disadvan- 
tage of a slower light response time. Through transfection with a viral vector con- 
struct in adeno-associated virus, high levels of recombinant melanopsin were 
introduced into the retinal cells of mice homozygous for the rd mutation, which 
results in complete loss of rod photoreceptors. The transduction of the rd mice with 
the ectopic melanopsin restored light response as determined by behavioral tests of 
live mice and light-stimulated action potentials of isolated retinal cells [39]. 



10.3.5 Nanoscale Photovoltaics: The Photosystem I Reaction 
Center 

As illustrated in Fig. 10.6, the photosynthetic membranes of green plants contain 
two molecular photovoltaic structures, Photosystems I and II (PSI and PSII) that are 
serially connected in an electron transport chain that drive the endergonic reactions 
of photosynthesis. The bioenergetic properties of PSI have been reviewed by 
Chitnis [5]. Photon absorption in PSI triggers a charge separation that generates a 
voltage across the photosynthetic membrane. This voltage is the source of Gibbs 
energy that drives the energetically uphill reactions of photosynthesis. It is possible 
to isolate PSI reaction centers and preserve their full photovoltaic properties [8, 15]. 
It has been proposed to fuse PSI reaction centers in membranes in close proximity 
to voltage-gated ions channels and to use the photovoltaic properties of PSI to gate 
these channels [10]. One example of the idea is illustrated in Fig. 10.7. PSI located 
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Fig. 10.6 Schematic illustration of the photosynthetic membrane. Photosystems 1 and II are inte- 
gral membrane nanoscale molecular photovoltaic structures. PSI can be used to impart photoactivity 
to mammalian cells 
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Fig. 10.7 Schematic illustration of PSI adjacent to a voltage-gated ion channel. One or more PSI 
reaction centers may be able to trigger the ion channels. Depending on orientation, PSI can depo- 
larize or hyperpolarize the membrane 



in close proximity to voltage-gated ion channels, either in the membrane or externally 
at the lipid-water interface, may be capable of generating a local electrical distur- 
bance of sufficient magnitude to create an excitatory postsynaptic potential and 
generate an action potential. Kuritz et al. [14] have shown that PSI-proteoliposomes 
incubated with retinoblastoma cells imparted optical activity to the cells as mea- 
sured by the light-induced slow movement of calcium ions into the cells. Pennisi 
et al. have performed experimental and theoretical studies on the incorporation of 
PSI reaction centers in human cells and lipid vesicles [25-27]. In particular, new 
methods of delivery and detection of PSI in the membrane of human cells have 
been developed (Fig. 10.8) [27]. Purified fractions of PSI were reconstituted in 
proteoliposomes that were used as vehicles for the membrane incorporation. 
A fluorescent impermeable dye was entrapped in the vesicles to qualitatively 
analyze the nature of the vesicle-cell interaction. After incorporation, the localiza- 
tion and orientation of the complexes in the membrane was studied using immuno- 
fluorescence microscopy. The results showed complexes oriented as in native 
membranes, which were randomly distributed in clusters over the entire surface of 
the cell. Additionally, analysis of cell viability showed that the incorporation pro- 
cess does not damage the cell membrane. Taken together, the results of this work 
suggest that the mammalian cellular membrane is a reasonable environment for the 
incorporation of PSI complexes, which opens the possibility of using these molecu- 
lar photovoltaic structures for optical control of cell activity. 

The direct use of chlorophyll itself as a visual pigment in animals has been 
observed in nature and attempted in the laboratory. Chlorophyll-like pigments have 
been found to be associated with the rhodopsin of deep sea fish [40, 41]. These 
pigments are believed to be used for dim light vision, as this would enable using the 
red part of the incident light spectrum, which has tenfold less loss due to scattering. 
Based on these observations, direct utilization of chlorophyll-derived pigments for 
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Fig. 10.8 Simultaneous evaluation of pyranine uptake and immunodetection of PSI in adipose 
tissue-derived stem cells. The images were obtained using differential interference contrasting 
(D1C; first row), and in the fluorescein isothiocyanate (F1TC) and Cy5 fluorescent dye channels 
(second and third row, respectively). In this experiment, pyranine fluorescence is detected with the 
F1TC channel and the secondary antibody fluorescence with the Cy5 channel. The images corre- 
sponding to the Cy5 channel clearly show that PSI complexes are associated with the membrane 
of the cells from the experimental sample in contrast with the controls. The fourth row has the 
merged images from the fluorescence channels, where it is possible to see that there is no overlap 
between the FITC channel (indicative of cytoplasmic localization) and the Cy5 channel (indicative 
of membrane localization). Individual cells that are separated from the rest and can be more 
clearly visualized are indicated with arrows in the DIC images. Scale bar = 30 urn. From [27], with 
permission. Biomedical Engineering Society © 2008 



enhancement of vision in mammals has been attempted [42]. Dark-adapted mice 
were injected with the water-soluble chlorophyll derivative chlorin-e 6 , resulting in its 
accumulation in the outer segment of the retina. Comparison of the response of 
chlorin-e 6 -injected and control live mice to red light indicated that they responded to 
red light. The electrophysiological response to red light of the melanopsin-expressing 
retinal ganglion cells was doubled in intensity compared to controls. 
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10.4 Synthetic Chromophores and Artificial Sight 

We presented a brief review of the three main techniques for imparting optical 
activity to mammalian cells: (1) light-mediated untethering (or "uncaging") of 
chemically modified signaling molecules; (2) chemical modification of ion channels 
and receptors to render them light-responsive and (3) introduction of light-sensitive 
proteins into nonphotoactive cells. These are post-electrode prostheses techniques 
and ideas in laboratory methodology for the study of excitable cells. In principle, they 
offer the ability to target multiple cells of a specific class simultaneously. External 
electrodes are limited in their spatial resolution for heterogeneous tissue. Although 
intracellular electrodes can target specific neurons, they don't lend themselves 
to simultaneous targeting of multiple cells of a specific subclass. Moreover, 
mechanical electrodes are intrusive structures in the context of excitable tissue. 
Intelligently designed molecular scale activators powered by photon absorption in 
synthetic chromophores can, in principle, blend into the membranes of excitable 
tissue with linear dimensions that are compatible with the scale-length and fine 
structure of the tissue. 

The field of synthetic chromophores and its application to artificial sight is moti- 
vated by advances that have been made with multielectrode retinal prosthesis 
arrays. Numerous studies have reported that stimulation of neurons in the visual 
pathway evokes the perception of light. It is assumed that analogous stimulation at 
the molecular level will mimic the action of electrodes, with the added advantage 
of nanoscale resolution and auto-power by the photons that trigger the neural activity. 
There is, at present, no clinical data to support this assumption. Moreover, in order 
for synthetic chromophores to be relevant to real- world applications they need to be 
stable or easily rejuvenated and operate at ambient wavelengths and light intensities, 
either on their own or in conjunction with optoelectronic signal conditioning 
devices. These are challenging areas of research and biomedical engineering that 
are currently in early stages of development. 
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Chapter 11 

Biophysics/ Engineering of Cortical Electrodes 

Philip R. Troyk 



Abstract This chapter provides a description of how microelectrodes are used to 
form an artificial interface to the cortex. Microelectrodes inserted into the cortex 
are called "intracortical electrodes" and are anticipated for use in cortical visual 
prostheses. Owing to the nature of the cortical environment, the design and use of 
these electrodes pose challenges for the clinical deployment of cortical prostheses. 
The combined effects of electrode charge injection and effects of the in vivo envi- 
ronment are discussed. 
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11.1 Background 

Electrical stimulation of the visual cortex has been used for investigative studies of 
the visual system and visual prostheses research since the first half of the twentieth 
century. These have all been based upon the biophysical phenomena of passing 
electrical charge within neural tissue for the purpose of activating the neural networks, 
and producing visual sensations commonly called phosphenes. For a visual pros- 
thesis, the underlying assumption is that the electrical stimulation can be organized 
into spatiotemporal patterns that can manipulate the stimulated neural substrate in 
a manner that exploits the natural tuning properties of the visual system. By strategic 
patterning of the stimulation, it is assumed that an image, captured by electronic 
means, can be translated into a visual sensation that mimics biological vision. To date, 
this assumption remains unproven. 

To a large extent, the primary limitation in designing, and deploying, all visual 
prostheses is the inability to implant an artificial neural interface that accommo- 
dates the density and scale of the visual system at the retina, optic nerve, or primary 
visual cortex. Despite the huge advances seen within the electronics industry, over 
the past 75 years, over this same time period the state-of-the-art for interfacing to 
neural tissue has not significantly changed. Electrical currents are passed into neural 
tissue through metal electrodes placed near the target neural tissue. 

For stimulation of the visual cortex, sub-dural electrodes have been previously 
used on the surface of brain, with limited success [7, 18-20]. Relative to surface 
electrodes, intracortical electrodes penetrate the visual cortex and use smaller elec- 
trical currents in closer physical proximity to the cortical neurons, and it is gener- 
ally accepted that their use, with exposed tip sizes on the order of the target neurons, 
has a significantly higher likelihood of success for the design of a cortical visual 
prosthesis. Compared to epi- and subretinal electrodes, implanted intracortical 
electrodes are surrounded by a very different medium, and this has contributed to 
notable differences in their in vivo behavior. Owing to their small size, and the need 
to stabilize their position within the cortical tissue, the functional understanding and 
mechanical design of intracortical electrodes have been particularly challenging. 



11.2 Physical Structure of Intracortical Electrodes 

Intracortical metal electrodes are typically fabricated from rigid shafts designed 
to penetrate the visual cortex. The shaft of the electrode is often insulated with a 
biocompatible polymer, e.g. Parylene-C. At the tip of the electrode is an exposed 
noble metal surface, commonly platinum or iridium. The surface area of the 
exposed tip is carefully controlled, during manufacture, so as to target a pool of 
neurons within a predefined semi-spherical volume surrounding the tip, while 
allowing for the safe transfer of charge. 

The material used for the shaft can vary. Silicon, polymers, and bare metal wires 
have been used. In its simplest form, the electrode is comprised of a metal wire 
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whose tip has been etched to a controlled-geometry point. More complicated structures 
use metal-tipped silicon shafts, and thin-film-fabricated multi-dimensional silicon 
shanks that contain multiple surface-deposited metal electrode sites [3, 44, 47]. 
The length of the electrodes is typically between 1.5 and 2 mm in order to target 
cortical neuronal layer IV. However, in practice it is difficult to assure the depth 
penetration, or maintenance of the tip position within the cortex. 

Singular discrete-wire intracortical electrodes have historically been used for 
both recording and stimulation of cortical neurons. Commonly, such electrodes are 
fabricated by cutting off the end of an insulated small diameter (25-50 urn) noble- 
metal (typically platinum-iridium) wire. More sophisticated designs use controlled 
etching of the bare metal wire in order to obtain a precise tip shape, polymer insulation 
of the electrode's shaft, and final laser ablation to precisely expose the metal tip. 
Positional stability of the singular intracortical electrode tip in the brain is crucial 
if it is desired to consistently record from, or stimulate, a particular neuronal pool. 
Gualtierotti and Bailey [24] are credited with being the first to describe a "neutral 
buoyancy" microelectrode. In their concept, the intracortical electrode needed to 
"float" on the surface of the cortex so that the normal movement of the brain would 
not disrupt the position of the electrode relative to the target neurons. Their design 
was not practical for mass production, however subsequent design have strived to 
retain this principle of minimal mass and mechanical floating. 

The "hat pin" intracortical electrode design was developed by Salcman and Bak 
[39], and was used in several human visual prosthesis investigative experiments 
[4, 40]. This design, a derivative of the Gualtierotti and Bailey electrode, is com- 
prised of a 37-u.m diameter iridium wire microwelded to a 25-u.m diameter gold 
wire lead, with the electrode tip etched to produce a radius of 1-5 urn. The elec- 
trode shaft is coated with a 3^1 |im thick layer of Parylene-C insulation. A dual-beam 
excimer laser is used to control the exposure of the metal tip. The junction between 
the electrode shaft and the connecting gold lead wire is reinforced with an epoxy 
ball, with the resulting structure resembling a hat-pin, as shown in Fig. 11.1. 
In some versions of this design, two electrodes are incorporated within a single 
epoxy ball to produce an electrode doublet. Insertion of the electrode structure into 
the cortex can be accomplished by hand, using surgical forceps. 

Other variations of this basic wire-type electrode design have evolved over the 
past 25 years that use blunter tips with controlled-cone shapes. In each case, the 
designers strived to produce electrodes that were consistent in shape, length, and 
tip exposure, with the goal of minimizing tissue insertion damage and preserving 
the underlying neuronal substrate. 

It has generally been accepted that a viable cortical visual prosthesis will 
require hundreds, possibly thousands, of intracortical electrodes, and while earlier 
experiments were successfully performed using singular intracortical electrodes, 
the surgical difficulty associated with implantation of individual electrodes moti- 
vated the design of electrode arrays. Cortical electrode arrays are comprised of a 
group of electrodes whose relative position and cortical penetration are maintained 
by a superstructure. The number of electrodes contained within an array can vary 
from 16 to over 100, depending upon the application and manufacturing method. 
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Fig. 11.1 Dual hat pin electrode as described by Salcman and Bak. From Schmidt et al. [40] 



Interconnection between the array's electrodes and electronic circuitry used for 
generating the stimulation currents becomes more challenging as the number of the 
electrodes within the array increases. 

Cortical insertion of the intracortical electrode arrays is most often performed 
using an array-specific high-speed insertion tool in order to minimize the "bed-of-nails" 
effect. Slow insertion of an array, through the pia, can cause unacceptable cortical 
deformation and significant micro-bleeding, thus damaging the target neuronal 
pool. These conditions are avoided by using rapid insertion, and depending upon 
the number electrodes within the array, and the electrode tip shapes, speeds from 1 to 
lOm/s are used. Using rapid insertion, the array can directly penetrate blood 
vessels with little to no resultant bleeding. 

A smaller electrode physical tip size offers the promise of selective stimulation 
of a small pool of neurons, and the design of the intracortical stimulation electrode 
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is most often faced with a compromise between the desire for a small-geometry tip, 
and a limitation in the charge per unit area (charge density) that can be safely 
injected into the tissue without causing damage to the electrode or the surrounding 
neuronal tissue. As the tip area is made smaller, the safe charge, and charge density, 
limit must be correspondingly reduced, albeit most often in a non-linear manner. 
Typically, intracortical electrodes are made with tip areas of less than 2,000 |im 2 , 
and for some studies electrodes as small as 200 urn 2 have been used [40]. 

Despite the variations in electrode shape, length, or array composition, the basic 
interface between the electrode and the neuronal tissues remains that of a metal 
surface through which electrical charge is passed, and it is the nature of this interface 
that occupies the efforts of numerous research laboratories. 



11.3 Charge Injection Using Intracortical Electrodes 
11.3.1 The Intracortical Electrode as a Transducer 

Electrical stimulation of neurons is accomplished by depolarizing the neuronal 
membrane through the flow of ionic current between two electrodes, and typically 
this is accomplished by injecting pulses of ionic current through the neuronal tissue 
that surrounds the electrode. Use of pulses to initiate the neuronal activation derives 
directly from the capacitive nature of the neuronal membrane. The fundamental 
function of the intracortical electrode is to act as a transducer for converting elec- 
tronic current, flowing from the stimulator circuitry to the electrode, into ionic 
current that flows within the biological tissues. In order to elicit a neuronal 
response, a threshold membrane polarization must be reached, and for a specific 
electrode geometry this defines a threshold stimulation charge that the electrode 
must support for each stimulation pulse. For a metal electrode to support the threshold 
charge injection, suitable processes occurring at the electrode-tissue interface are 
required to cause the necessary charge-carrier conversion. These processes can be 
capacitive, or faradaic. In the former, a capacitive interface, formed by either the 
electrode-electrolyte double-layer, or a dielectric layer, is charged and discharged. 
For faradaic reactions, electrochemical reactions involve metal-specific charge 
species that are oxidized and reduced. In order to protect the electrode and the 
surrounding tissue from deterioration, these reactions must be reversible and limit 
the injection of reaction by-products into the tissue. 

Capacitive-type electrodes [25], inject charge exclusively through capacitive 
charging and discharging, and therefore they are conceptually attractive, for use as 
intracortical electrodes, because they avoid the use of faradaic reactions. However, 
presently-known capacitive electrodes do not have sufficiently high charge-storage 
capacities to make them useful for intracortical stimulating electrodes. The use 
of surface-roughening, porous-material electrode coatings, and high-dielectric con- 
stant films, such as Ta,0 5 or Ti0 2 , have been investigated in an attempt to increase 
capacitive electrode charge capacities to the required stimulation threshold levels. 
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However, even with these techniques, the charge injection capacity of current 
capacitive-type electrodes is not adequate for small area (<2,000 u.m 2 ) intracortical 
electrodes [37, 38]. 

Faradaic-type electrodes, accomplish the charge-carrier conversion through 
reduction and oxidation (redox) of surface species. One metal widely used for 
numerous stimulation electrodes, including intracortical microelectrodes, is Platinum 
(Pt) or a Pt-based alloy. While the precise nature of the reactions utilized by a Pt 
electrode is unclear, it is generally accepted that charge transfer occurs substantially 
by H-atom plating and stripping, with the double-layer capacitance contributing less 
than 15% to the total charge injection - Fig. 11.2. To improve the ability of an 
electrode to act as a transducer and inject higher levels of ionic current into the 
tissue, coatings such as activated-iridium-oxide-film (AIROF) have been used. 

In order to initiate the faradaic reaction, a reaction-activation voltage drop across 
the metal-tissue (metal-electrolyte) interface must be achieved, and this voltage 
drop is commonly called the "electrode polarization." The voltage drop is caused by 
the flow of current, electronic and ionic, at the metal-electrolyte interface. Initially, 
the current flow, and charge transfer, is supported by the charging, or discharging, 
of the double-layer capacitance. As the charge in this capacitance is quickly 
exhausted, the resulting increase in electrode polarization initiates the first available 
redox reaction capable of supporting the charge injection. As this initial reaction 
is exhausted, either due to unavailability of counter ions or reaction rate limitations, 
a new reaction must be recruited. The surface conditions of the metal electrode 
might cause the initiation and exhaustion of several reactions as the polarization 
increases and the charge injection is continuously supported. 

In theory, the charge injection process is reversible under the assumption that 
the redox reactions which are utilized are reversible. Therefore, it is common to use 
two phases for the stimulation waveform. The first phase is designed to stimulate the 
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Fig. 11.2 Depiction of charge injection reactions taking place at the surface of Iridium Oxide 
(top) and Platinum (bottom) intracortical electrodes. The AIROF film provides a buffer zone in 
which reversible redox reactions can take place and permit enhanced charge injection 
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target neuronal pool. The second phase is designed to reverse the faradaic reactions 
that were utilized in the first phase by reversing the current in the electrode-electrolyte 
interface. Most often, but not exclusively, stimulation waveforms are generated by 
electronic current sources with the magnitude and duration of the first phase being 
replicated with an opposite magnitude and identically-timed pulse in the second 
phase. Owing to the simplicity in producing rectangular current pulses, the biphasic- 
balanced-constant-current rectangular stimulation waveform has become an historical 
standard for most neural stimulation systems [21, 22]. In order for an intracortical 
electrode to act as a stimulating transducer for chronic stimulation of neural tissue, 
the reversibility of the reactions must be assured, or else redox reaction by-products 
and unacceptable local pH shifts may damage the tissue and render the surrounding 
neuronal pool unusable [2, 9, 14, 36]. While some biological damage is unavoid- 
able, due to mechanical damage resulting from the electrode insertion and electrical 
damage resulting from continuous charge injection, the damage must be self-limit- 
ing in order for the intracortical electrode to be a critical component for a cortical 
stimulation system. 



11.3.2 Charge Injection Limits 

Unfortunately, the total charge capacity of the reversible reactions that are available 
for a bare Pt intracortical electrode is often insufficient to initiate the necessary 
neuronal response. While Pt has been used quite successfully for heart pacers, 
cochlear implants, and other implanted neural stimulators, the extremely small size 
of intracortical electrodes limits the useable electrode charge capacity to well below 
the charge-per-pulse required for stimulation of cortical neurons. Despite this, it is 
quite easy to continue driving a Pt electrode with electronic current past the point 
at which the reversible reactions are exhausted, thus necessitating the recruitment 
of irreversible reactions. The most frequently recruited irreversible reactions are 
oxidation and reduction of water. From an electrochemical standpoint, water 
decomposition can provide an almost inexhaustible supply of charge carriers within 
the tissue, albeit with corrosion of metal. From a biological standpoint, decomposi- 
tion of water as a means of neural stimulation is accompanied by huge local pH 
shifts, migration of metal ions into the tissue, and evolution of hydrogen and oxygen 
gasses. It is considered unacceptable to initiate water decomposition during cortical 
neural stimulation. Therefore, considerable research has been dedicated to under- 
standing how to avoid the use of irreversible reactions during neural stimulation. 

It is well-known that for a given electrode design, consisting of a particular 
metal type and geometric shape, there exists a window of polarization for which the 
electrode will not experience water redox reactions. This window is commonly 
called the "water window," and, for many metals, is roughly within the range of 
+0.8 to -0.6 V with respect to a Ag|AgCl reference electrode. As long as the elec- 
trode polarization is held to a value within the water window there is a reasonable 
expectation that water decomposition will not occur. This should not be interpreted 
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to mean that all reactions occurring within the water window are safe: There is 
some suspicion that for many platinum electrode designs, some reactions taking 
place within the water window are not entirely reversible. 

Understanding the conditions that drive an electrode's polarization outside of the 
water window is often difficult, and for many years researchers attempted to define, 
a priori, the maximum charge density per stimulation pulse that a particular metal, 
or electrode coating, could support. This quantity became known as the maximum 
injectable charge density and is expressed in units of charge/area, frequently using 
mC/cm 2 . Establishment of the maximum injectable charge density for a particular 
electrode design is difficult due to the uncertainties about the relationship between 
charge injection and electrode polarization. Often, limits were defined based upon 
empirical in vitro studies, in which the physical condition of the electrode was 
examined following an extended pulsing regime [6]. In companion studies, elec- 
trodes were pulsed in vivo using predefined charge densities with post implantation 
histology being performed upon the surrounding tissue to examine adverse effects 
upon the local neurons or migration of metal ions [29]. Unfortunately, studies for 
particular electrode designs, using particular electrode metals or coatings, were 
often extrapolated to define the general charge injection limits for that same metal, 
or coating, type without regard to the electrode geometry. This created much confu- 
sion about how electrodes could be designed and used for chronic cortical stimula- 
tion so as to avoid both deterioration of the electrode and damage to the surrounding 
biological tissue. 

In addition to damage that might be imparted to the surrounding tissue from the 
use of irreversible reactions, neurons can also be adversely affected, or even dam- 
aged, by over-excitation, even when operating an electrode within the water window. 
These changes in neuronal sensitivity and viability were observed through studies 
at the Huntington Medical Research Institutes [29]. From a functionality stand- 
point, over excitation of cortical neurons can produce a depression in neuronal 
activity and sensitivity. Once they are excessively stimulated, the neurons become 
less sensitive to subsequent stimulation pulses, thus shifting the firing threshold of 
the target neuronal pool. Depression can occur from either individual electrodes, 
near a small pool of neurons, being driven above the charge-induced depression 
level, or by ensembles of electrodes whose individual sub-depression charge injec- 
tion levels summate causing distributed depression within a larger neuronal pool. 
In general, histological studies of depressed neurons do not show physical damage 
[29]. Yet, recovery of the depressed neurons can require hours, or even days, after 
cessation of the stimulation. Beyond the depression-induced threshold, physical 
damage to cortical neurons has been observed as a result of stimulation pulsing 
even when there is no evidence of driving the electrode beyond the water window 
[30]. This condition results in a much more serious impact upon the target neuronal 
pool since it is irreversible. 

The combined effects of reaction-induced electrode or tissue damage and the 
stimulation-induced tissue depression or damage are hard to quantify. To date, no 
studies have established a priori charge injection limits for intracortical electrodes 
that consider both electrochemical and stimulation-induced effects. Furthermore, 
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the temptation to apply results from studies of one electrode design to other 
electrode designs that use the same metal or coating often results in the discovery 
of electrode failure, or damage to the surrounding neurons, only after the elect- 
rode is implanted and chronically stimulated. Using the stability of the functional 
response of the stimulated neurons as a measure of safe charge injection is risky, 
since shifts in neuronal thresholds may occur only after damage to either the elec- 
trode or the tissue has already occurred. 



11.4 Intracortical Electrode Coatings 

Recognizing the charge injection limitations of intracortical metal electrodes, the 
National Institutes of Health, under the administration of the Neural Prosthesis 
Program, funded research to identify coatings that could be placed over metal elec- 
trodes towards the goal of limiting the charge injection reactions to within the water 
window [30]. This resulted in the identification and development of Activated 
IRidium Oxide Film (AIROF) at EIC Laboratories (Norwood, MA). AIROF is a 
faradaic coating that is based upon the electrochemical growth of a three-dimensional 
film of hydrated iridium oxide [1, 30]. Presently, AIROF, and Sputtered IRidium 
Oxide Film (SIROF) [15, 27, 41] have emerged as the preferred coating materials 
for many neural stimulating electrodes. 

AIROF is formed upon pure iridium metal electrodes using an electrochemical 
activation process. The attractiveness of AIROF as a stimulation electrode coating 
was recognized by Brummer and first reported in 1983 [8, 35]. Charge injection 
limits for AIROF electrodes in physiological buffer were reported in 1988 by Beebe 
and Rose [6]. 

The electrochemistry of the activation process has been studied extensively and 
models to explain the observed accumulation of oxide and the charge propagation 
mechanism have been suggested [10, 11, 30, 32]. It is known that thick anodic 
oxide films can be formed on the surface of an iridium electrode by continuously 
cycling the electrode potential with a triangular or square waveform in an aqueous 
electrolyte. The potential limits are typically between values slightly positive of 
hydrogen evolution and just negative of the onset of oxygen evolution. It has been 
shown that AIROF formation is influenced by the chemical composition of the 
electrolyte; the geometry and morphology of the iridium metal substrate; and, the 
duration and form of the voltage/current activation waveform [23, 45]. The electro- 
lyte composition influences the rate of formation as well as oxide morphology 
through the pH, the ionic strength, the conductivity and structure of the double 
layer at the Ir/electrolyte interface [17]. 

The benefit of using AIROF on intracortical electrodes stems from the premise 
that the redox reactions needed for charge transfer from the electrode to the tissue 
can take place exclusively within the film. Thus the film serves as a buffer zone for 
charge transfer between the metal surface and the biological electrolyte. AIROF is 
known for demonstrating significantly higher maximum charge injection limits, 
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and this high charge capacity is obtained from a reversible Ir 3+ /Ir 4+ valence transition 
that takes place within the film [8, 35] as depicted in Fig. 11.2. By restricting the 
redox reactions within the film, and utilizing a known high-charge capacity reaction, 
an increase in the injectable charge capacity and significantly improved safety and 
consistency of neural stimulation can be obtained. For cathodal-first stimulation 
pulses, the ability of AIROF to inject charge can be further be increased by applying 
a positive bias of 0.4-0.8 V (vs. Ag|AgCl) prior to the stimulation pulse [6] . The bias 
acts to convert the AIROF from a mixed Ir 3+ /Ir 4+ valence state to the Ir 4+ valence 
state, not only making the film significantly more electronically conductive, but also 
richer in the Ir 4+ needed for reduction during the cathodal phase. In some cases, the 
use of bias allows for as much as a factor of three increase in charge capacity [16]. 

It is not surprising that iridium oxide films have emerged as the preferred coat- 
ings for intracortical, and other neural prosthesis, electrodes. AIROF has been 
shown, in vitro, to allow for about 10-20 times the maximum injectable charge, 
when compared to bare Pt, achieving a charge density limit of up to 3.5mC/cm 2 
for anodally -biased cathodal-first pulses [6]. However, the use of AIROF, rather 
than bare metal, is fraught with some additional peril. AIROF is susceptible to 
damage if the electrode polarization moves outside of the water window. Initiating 
water decomposition reactions can cause the AIROF to delaminate from the 
underlying metal surface, thus rendering the electrode non-usable for continued 
charge injection. While it is generally regarded that it is not viable, for any elec- 
trode, to inject charge outside of the water window, there is often uncertainty about 
the voltage and current conditions for which the electrode polarization exceeds the 
water window limits. If using a Pt electrode, a momentary transgression of the water 
window limits may cause highly undesirable reactions and residual by-products 
that enter the tissue. Yet the surface of the electrode may remain relatively 
unharmed. For the AIROF electrode, the films acts as a buffer zone that protects 
the tissue, and therefore reactions outside of the water window can potentially 
damage the AIROF in an irreversible manner. 



11.5 Characterization of Intracortical Electrodes 
11.5.1 Cyclic Voltammetry 

Since the faradaic reactions used for electrode/tissue charge transfer are initiated by 
polarization of the electrode-electrolyte interface, it is useful to use an analytical 
method for examining how this interface behaves within, and outside of, the water 
window. Cyclic voltammetry (CV) is a commonly-utilized method for accessing 
the nature and behavior of stimulating electrodes. As derived from standard elec- 
trochemical methods, CV uses three-electrodes within an electrolyte. The potential 
of the intracortical electrode, with respect to a reference electrode, is periodically 
swept between two predetermined potential limits, usually at the water window 
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boundaries, while measuring the current that flows between the intracortical electrode 
and a larger counter electrode. The potential sweep shifts the electrode-electroyte 
interface through the full range of reversible redox reactions while the measured 
current provides an indication of the capacity and rate of these reactions. Integration 
of the CV current waveform is often used to calculate a total charge storage capacity 
(CSC), for both anodic-(CSC A ) or cathodic-(CSC c ) first stimulation. Typically, CV 
electrode measurements are made at sweep rates that are much slower than the volt- 
age changes which the electrode-electrolyte interface experiences during a typical 
stimulation pulse, with CV sweeps typically on the order of 50mV/s. Since the 
charge injection redox reactions are rate dependent, it is important to understand 
that CSC values are always larger than maximum charge injection values for any 
given electrode. Typically, less than 20% of the CSC can be utilized during a stimu- 
lus pulse. In this regard, review of the literature can often become confusing when 
comparing reported values of CSC to reported values of charge injected in vitro and 
in vivo. In other words, only a fraction of the CSC can be accessed during a short 
duration stimulus pulse. The CV measurement is highly sensitive to the condition 
of the electrode-electrolyte interface, the morphology the electrode coating, the 
electrode surface roughness, the geometric shape of the electrode tip, and the nature 
of the electrolyte. For any electrode metal, or coating, the shape of the CV can vary 
dramatically, depending upon how the electrode is fabricated and in what electro- 
lyte the measurement is performed, even though the nature of the redox reactions 
themselves remains the same. 



11.5.2 Electrode Stimulation Voltage Waveforms 

Stimulation of cortical neural tissue is most commonly accomplished by driving the 
electrode with a two-phase waveform that consists of a first neural-stimulation 
phase and a second charge-recovery phase. Typically, each of these phases are gen- 
erated by constant-current electronic circuits producing rectangular pulses. Often 
the first phase consists of a cathodal (negative) constant current pulse, followed by 
a second phase anodal (positive) constant current pulse as depicted in Fig. 11.3. 
In Fig. 11.3, a highly simplified model for an intracortical electrode is presented 
consisting of a series resistive-capacitive network. While simplistic, this model 
does allow for a first-order understanding of the relationship between the electrode- 
electrolyte interface and the voltage/current waveforms. The resistive component is 
commonly called: the access resistance, and the capacitive component is commonly 
called: the electrode pseudocapacitance. These are, of course, merely lumped- 
model approximations for the electrical and electrochemical processes that take 
place during a stimulation pulse. 

Referring to Fig. 11.3, during the first cathodal phase, constant current is 
forced through the electrode for the purpose of activating near-by cortical neu- 
rons. At the leading edge of the current pulse, an immediate voltage drop across 
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the electrode-electrolyte interface is observed. For this simplified model, this 
leading-edge drop is caused by the IR drop on the access resistance. In accor- 
dance with circuit theory, the voltage on the capacitor remains unchanged at the 
current pulse leading edge. As current is forced through the electrode, the capaci- 
tance, C, charges in a time-linear manner, deriving from /= C(dvldt). In the model 
of Fig. 11.3, the charging of this capacitor (dv) represents the electrode polariza- 
tion, and the redox reactions should remain within the water window provided 
that dv< 0.6 V. 

At the end of the first phase, the current changes from cathodal to anodal as the 
second charge-recovery phase is initiated. For the lumped model, the magnitude of 
the first-phase trailing edge step of the voltage waveform is twice that of the first- 
phase leading edge because the summation of the turning off of the cathodal current 
and the turning on of the anodal current produce a current step of twice that of the 
leading edge. This voltage step is the drop across the access resistance, R. During 
the second phase, anodal current is forced through the electrode in an attempt to 
restore the electrode to the pre-stimulus condition. In the simple model of Fig. 11.3, 
use of equal (but opposite) first and second phase currents, with equal pulse dura- 
tions, produces equal first and second phase charges, thus exactly returning the 
electrode to the pre-stimulus voltage level in anticipation of the next stimulus pulse. 
During the interval between biphasic stimulation pulses, some method of electrode 
voltage control is typically employed to assure that the electrode potential remains 
stable, at a pre-determined level, so that for repeated stimulation pulses the electrode 
can stimulate neurons in a consistent manner. 

In practice, the simplistic model of Fig. 11.3 fails to account for important 
aspects of the electrode's charge injection process. These include: (1) Multiple 
contributions to the access resistance drop that are inconsistent with an ideal resistor 
model, (2) Non-linear behavior of the electrode polarization that is inconsistent with 
an ideal capacitor model, and (3) Imbalances in the stimulator phase charges. 



11 Biophysics/ Engineering of Cortical Electrodes 219 

11.5.3 Non-ideal Access Resistance Behavior 

Historically, the leading edge voltage drop was attributed to the electrolyte resistance 
caused by limitations in ionic conductivity of the electrolyte. Thus it was common 
practice to subtract the entire leading edge drop from the total electrode voltage excur- 
sion, during the first phase, as a means of determining the electrode polarization. 
However, the leading edge drop can include other effects besides simple electrolyte 
resistance, specifically, concentration polarization near the electrode-electrolyte inter- 
face. Concentration polarization is essentially caused by a depletion in electrolyte 
charge carriers (counter ions) at the onset of the current pulse. For coated metal elec- 
trodes, such as AIROF, near instantaneous changes in film conductivity at the leading 
edge of the current can be a secondary contribution to the access voltage drop. 



11.5.4 Non-linear Electrode Polarization 

Based upon the earlier discussion, it is obvious that the dynamics of charge injection 
via redox reactions cannot be directly compared to the charging and discharging of 
an ideal capacitor. Owing to the complex geometric shape of the electrode tip, and 
the highly non-uniform current densities, as well as the range of possible of redox 
reactions that might be experienced, the behavior of the electrode-electrolyte inter- 
face might be better explained by a set of distributed RC networks, however even 
this remains an oversimplification. Rather, the behavior of the electrode during what 
is often called the electrode polarization phase, or the capacitive charging phase, is 
driven by the rates of one or more reactions, the changes in interfacial and film con- 
ductivity, and the closeness of the electrode voltage to the edge of the water window. 
Strictly speaking, the electrode polarization is comprised of the reaction activation 
overpotential and a shift in the electrode equilibrium potential. However, these com- 
ponents cannot be easily derived from the stimulus voltage excursion waveform. 



11.5.5 Determining Electrode Safety 

The uncertainties in determining the components, and magnitude, of the leading-edge 
access voltage drop make the estimation and prediction of electrode polarization, 
during any given stimulus pulse, difficult. Often the leading edge drop is by far the 
largest component of the total voltage excursion experienced by an electrode during 
a stimulation pulse. Simply subtracting the measured access voltage from the total 
voltage excursion is most often inadequate for estimating whether the electrode 
polarization is within the water window. It is unclear how much of the leading edge 
access voltage drop is truly caused by a benign resistive drop, and how much is 
caused by an interfacial process that might contribute to undesirable redox reactions. 
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Fig. 11.4 Depiction of an alternate current waveform for delivering stimulation pulses to 
microelectrodes. During the zero-current interphase portion of the waveform the effect of access 
resistance upon the voltage waveform is eliminated, and the residual voltage, during the inter- 
phase, is a reasonable measure of the electrode polarization. To remain within the water window 
the measured polarization should be more positive than -0.6 V with respect to Ag|AgCl 

Too often, the injectable charge capacity of a particular electrode is estimated from 
the use of a priori published material type-based charge densities, and this approach 
does take into account the actual dynamic behavior of the electrode since the wrong 
parameter, i.e. charge density rather than electrode polarization, is being consid- 
ered. An alternate method of estimating the electrode polarization is depicted in 
Fig. 1 1.4 and involves adding a third interphase region to the stimulation waveform. 
If a short period of zero-current is imposed between the first and second phases, 
then a measurement of the electrode voltage during this time of zero-current should 
be free from true IR drops, and should be a better estimate of the polarization 
caused by the delivery of charge during the first phase of the biphasic waveform 
[13]. The disadvantage to this approach is that the measurement is made after the 
polarization has occurred. There exists some debate about whether a typical AIROF 
intracortical electrode can tolerate single-stimulus conditions that transgress the 
water window without damage, and use of the interphase voltage measurement as 
a continuous measure of electrode safety may be inadequate to protect an AIROF 
electrode. However, there presently exists no implantable stimulator that uses 
leading-edge voltage measurements in a predictive manner to protect either AIROF, 
or bare metal, intracortical electrodes from damage. 



11.6 Contrasts of In Vitro and In Vivo Behavior 



Most available data for intracortical electrodes come from in vitro studies that were 
carried out in model physiological fluid. Based upon those studies, the maximum 
injectable charge capacity for AIROF intracortical electrodes whose tip areas are 
under 2,000 urn 2 is well within the anticipated stimulation charge thresholds for visual 
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cortex neurons. For example, Schmidt et al. [40] observed that stable phosphenes 
could be obtained in a human volunteer when using 0.4-4.6 nC/phase of stimula- 
tion, whereas in vitro measurements of 2,000 um 2 AIROF electrodes in phosphate- 
buffered saline typically show up to ten times this required charge capacity, while 
maintaining operation within the water window. This led to the historical conclu- 
sion that AIROF intracortical electrodes were more than adequate for long-term 
stimulation of the visual cortex in visual protheses. 

More recently, this view has been challenged, as in vivo studies of AIROF and 
SIROF electrodes have been performed. Cogan et al., compared the in vitro and 
in vivo charge injection behavior of large area (-125,000 |im 2 ) AIROF electrodes 
intended for a retinal visual prosthesis [12], and found that the charge capacity of 
the electrodes, once implanted subretinally in rabbits, required three times the total 
electrode voltage excursion as had been observed in vitro, for delivery of the same 
charge. Hu et al. [26] compared the performance of 2,000 urn 2 intracortical elec- 
trodes implanted within the cortex of a zebra finch with their performance in dilute 
phosphate buffered saline, and found a factor of four decrease in their charge injec- 
tion capacity in vivo for equal in vitro and in vivo voltage excursions. Even more 
disturbing, is the observation that electrode polarization, in vivo, appears to increase 
by a factor of two, over that seen in vitro, for equal charge injection [12]. 

Figure 11.5 shows a dramatic demonstration of the loss of charge capacity, 
relative to in vitro behavior, for intracortical electrodes placed within the in vivo 
cortical environment. Two electrodes were measured in vitro, immediately placed 
in vivo, then immediately replaced into the in vitro environment. The stimulator 
circuitry was specially designed to limit the total electrode voltage to less than 
±0.6 V (water window) in order to prevent electrode damage. On the left of 
Fig. 11.5 are shown the pulse voltage excursions for the two AIROF intracortical 
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Fig. 11.5 AIROF intracortical electrodes tested in vitro and in vivo. Electrodes were transferred 
between a beaker of PBS and the cortex of a Zebra Finch during the same experiment. In vitro 
current and voltage excursions for two electrodes are shown on the right and left set of plots. 
In vivo waveforms are shown in the center plots. Note the dramatic decrease in the in vivo injectable 
charge capacity, relative to the in vitro behavior, as seen in the center plots by the larger voltage 
excursions for the smaller stimulation currents 
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electrodes, of area 2,000 |im 2 , while placed within phosphate-buffered saline. 
The voltage and current are indicated on each of the plots as (I) and (V). On the left, 
it can be seen that for approximately -0.6 V of total voltage excursion, the electrodes 
are able to support -75 uA for 300 |is (22.5 nC). In the center plots, the same two 
electrodes are shown placed within the cortex of a Zebra Finch. Note that for the 
same voltage excursion as was seen, in vitro, in vivo less than lOuA is supported 
for 300 |is (3 nC), and the leading edge access voltage has significantly increased. 
On the right one can see the two final in vitro plots, with the charge capacity 
restored to the original in vitro values. 

These studies, and others, lead to the conclusion that there are notable differ- 
ences between the in vitro and in vivo environment that significantly impact the 
ability of an intracortical electrode to act as a charge transfer transducer for a corti- 
cal visual prosthesis. While the behavioral differences with respect to electrode 
voltage excursion seem self-evident, the causes of them remain unclear. Yet, there 
are identifiable and unique characteristics of the in vivo environment: (1) The pres- 
ence of proteins and other organic species, (2) The presence of cells that could 
impede the mobility and transport of counter ion charge carriers, (3) Encapsulation 
of the electrode surface by bio-molecules. For the Zebra Finch experiment of 
Fig. 1 1.5, described above, there was insufficient time for cell growth or significant 
encapsulation of the electrodes. Indeed, only minor washing of the electrode was 
performed when transferring from the in vivo to the in vitro environment. One 
hypothesis is that of the three factors listed above, reduced mobility of counter ions 
seems the most likely. More studies are needed in order to explain, and understand 
how to minimize the adverse effects of the in vivo environment upon electrode 
charge injection. Whatever the causes, the capabilities of present-day intracortical 
electrode technology seem marginal, but probably adequate, for the demands of 
current cortical visual prostheses designs. However, future visual prosthesis designs 
may very well exceed currently-achievable injectable charge capabilities. 



11.7 Alternative Coatings for Improving Intracortical 
Electrodes 

11.7.1 SIROF 

Iridium oxide films can also be applied to metal surfaces using reactive ion sputtering 
within an oxidizing plasma [15, 27, 41, 43]. The performance of SIROF in vitro and 
in vivo favorably compares to that of AIROF One advantage of the SIROF is the 
physical robustness of the deposited films, and this might make them more resistant 
than AIROF to short-term operation outside of the water window. Additionally, the 
ability to sputter the films on a variety of base metals while using masking, or other 
patterning techniques might make SIROF more adaptable to a wider range of physical 
intracortical electrode designs. 
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11.7.2 PEDOT 

Polyethylenedioxythiophene is an emerging electrode coating material that is based 
upon an electrically conducting polymer. Earlier work on other electrically conducting 
polymers was less than promising, however more recently there has been the sug- 
gestion that such polymers might be modified with biomolecules or nerve growth 
factors in order to promote the functionality of the interface between the electrode 
and surrounding neurons. In theory, if neurons could be attracted nearer to the elec- 
trodes than is ordinarily seen, lower stimulation thresholds might be achieved. 
PEDOT-coated electrodes, that are characterized by reduced impedances have been 
used for chronic recording studies [28]. Preliminary work towards the possibility of 
performing in vivo polymerization of PEDOT has been reported [33, 34, 46]. Charge 
injection capabilities of PEDOT, in vitro, are on the order of AIROF [31], although 
in vivo studies are currently lacking. The nature of charge injection by PEDOT with 
respect to capacitive or faradaic means has not been fully explained. There is some 
suggestion that for potentials more positive than -0.6 V, the primary process may be 
capacitive although present studies are not conclusive. 



11.7.3 Carbon Nanotube Coatings 

Carbon Nanotubes have been suggested as an intracortical electrode coating due to 
the extremely high increase in surface made possible by the presence of the tubes 
[42]. This work is in the early stages of research. The basic principle is that by 
dramatically increasing the surface area of the electrode, via the three dimensional 
structure, a corresponding increase in the double layer capacitance would result. 
Earlier attempts to use more conventional means of surface roughening and cre- 
ation of porous structures for metal electrodes showed similar increases in surface 
area and in vitro charge injection. However, once placed into the biological environ- 
ment adsorption of biomolecules seemed to clog the pores and defeat the strategy. 
Whether carbon nanotubes would be susceptible to similar effects is presently 
unknown. It may be possible to chemically alter the nanotubes so as to improve 
their biocompatibility [5]. 



11.8 Conclusion 

Present-day technology for fabricating intracortical microelectrodes still relies 
upon mechanically stabilizing a metal, or a coated metal surface, near a pool of 
target neurons. In this regard, the basic technology for transferring charge to cortical 
tissue has not changed within the past century. A better understanding of how a metal 
electrode functions as a charge-transfer transducer has facilitated the establishment 
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of relatively safe driving strategies for chronic stimulation of cortical neurons. 
Whether this technology is sufficient for the deployment of a cortical visual prosthesis 
remains untested. The sensory functionality demonstrated by cortical prostheses that 
used larger electrodes on the surface of the brain was disappointing from the stand- 
point of the users' abilities to integrate stimuli into coherent visual perceptions [18]. 
Despite some of the limitations in current electrode technology, it is expected that 
modern versions of these earlier surface electrode visual prostheses, that utilize 
intracortical electrodes, may be used in human trials within the next 5 years. 
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Chapter 12 

The Response of Retinal Neurons 

to Electrical Stimulation: A Summary 

of In Vitro and In Vivo Animal Studies 

Shelley I. Fried and Ralph J. Jensen 



Abstract The studies reviewed in this chapter are restricted to those that electrically 
stimulate the retina. The research studies reviewed in this chapter are further limited 
to those performed in animal models; the results of human clinical studies are covered 
in subsequent chapters. 

The neural response to electrical stimulation is influenced (potentially) by a large 
number of stimulation-related variables (Chaps. 6-10). Stimulating electrodes can 
be constructed in different shapes and sizes and fabricated out of different materials. 
Arrays of multiple electrodes can be configured in many different arrangements 
and ultimately positioned on opposite sides of the retina, or even penetrate into the 
retina. In addition, the phase length, duration, amplitude and/or frequency of stimulus 
pulses can each vary, some by several orders of magnitude. 

The neurobiology of the retina creates additional variables. There are five major 
classes of retinal neurons and each is a potential target of electrical stimulation. 
Each class can be subdivided into many different types; the anatomical and bio- 
physical properties of each can vary considerably. Therefore, the response to elec- 
trical stimulation may also vary across types. Since each type is thought to convey 
different features of the visual world, stimulation methods that do not activate all 
types appropriately may not convey some or all of the important features of the 
visual scene. 

Systematic study of the interactions between all engineering and neurobiological 
variables requires an extensive matrix of experiments. As a result, many basic 
questions remain unexplored. Here, we will focus on the experimental studies that 
have yielded the more important insights into either the mechanism by which retinal 
neurons respond to electrical stimulation or those that have led to improved 
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stimulation methods. The final section of this chapter is devoted to a discussion of 
some important, unanswered questions. 



Abbreviations 

AMD Age-related macular degeneration 

AP4 2-Amino-4-phosphonobutyric acid 

CNQX 6-Cyano-7-nitroquinoxaline-2,3-dione 

DS Directionally selective 

EECP Electrically elicited cortical potentials 

LED Local edge detector 

LFP Local field potential 

MK801 (+)-5-Methyl-10,ll-dihydro-5H-dibenzo[a,d]cyclohepten-5,10-imine 

maleate 

NBQX 2-3-Dioxo-6-nitro- 1 ,2,3,4-tetrohydrobenzo[f]quinoxaline-7-sulfonamide 

RCS Royal College of Surgeons 

Rdl Retinal degeneration 1 

RGC Retinal ganglion cell 

RP Retinitis pigmentosa 



12.1 Introduction 

Over 10 years ago, Humayun et al. [17] demonstrated that electrical stimulation of 
the retina elicits light percepts, called phosphenes, in patients that had been blind 
for many years. This remarkable finding has since been duplicated by other research 
groups [11, 41, 60]. However, the size, shape, color and contrast of phosphenes 
vary considerably, and, despite considerable effort over the last decade, only limited 
improvements in quality and consistency have been reported [18, 41]. 

The reasons underlying the lack of consistent, high-quality percepts are not well 
understood. While many factors are likely to contribute, one significant factor 
is thought to be the disparity between the neural activity elicited by the prosthesis 
vs. the neural activity that arises normally in the healthy retina [42, 58]. Prosthetic 
elicited activity that is too non-physiological may simply be unintelligible to the 
brain. Although perfect replication of physiological signals is well beyond current 
technology, it seems intuitive that the closer elicited patterns come to matching 
physiological patterns, the better the elicited vision will be. 

Methods to create specific patterns of neural activity will arise from a solid 
understanding of the basic interactions between electrical stimulation and retinal 
neurobiology. The challenge therefore is to improve our understanding of how and 
why retinal neurons respond to stimulation. An increasing number of in vitro and 
in vivo research studies have begun to analyze the responses of retinal ganglion 
cells (RGCs) to electrical stimulation and we are now beginning to understand 
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some of the more basic mechanisms by which neural activity is generated. The 
focus of this chapter is to review the progress in this area. 



12.2 Responses of RGCs to Electrical Stimulation 
in Normal Retina 

The two most common retinal implant configurations are referred to as sub- and 
epiretinal; the primary difference being the location of the stimulating electrodes 
(Fig. 12.1). Epiretinal electrodes are positioned on the innermost surface of the 
retina; ideally, they are in close proximity to RGCs and the nerve fiber layer. 
Subretinal electrodes are placed in the outer retina, close to bipolar cell dendrites 
when used in degenerate retina (but close to photoreceptors in studies that use nor- 
mal retinal tissue). In this section we will examine the studies on epiretinal and 
subretinal stimulation of normal retinas; later sections will focus on similar studies 
in degenerate retina. 



12.2.1 Epiretinal Stimulation 

12.2.1.1 Target of Stimulation 

Surprisingly, the duration of the stimulus pulse is the key parameter that determines 
which class of retinal neuron is activated by epiretinal stimulation [1, 9, 12, 24, 28]; 
an idea first reported by Greenberg [12]. Short duration pulses, -0.1 ms, elicit only 
a single action potential, typically within 1 ms after the onset of a cathodal pulse 
[9, 23, 49]. In some cases, the response to a short pulse consists of a spike doublet - 
the latency of the first spike is <1 ms while the latency of the second spike varies 
between 5 and 15 ms [49]. These responses persist in the presence of synaptic 
blockers indicating that they arise from direct activation of RGCs. 



NFL 



GCL- 
INL 

PRL 



illl 



[ 



Epi-retinal electrodes 



^L 



!£ 



,o lo 

O i c 



1 



Sub-retinal electrodes 



Mil 



Fig. 12.1 Placement of sub- vs. epiretinal electrodes. Epiretinal electrodes are placed on or near 
the innermost surface of the retina while subretinal electrodes are positioned in the outer retina - 
approximately at the location once occupied by the photoreceptors. NFL nerve fiber layer, GCL 
ganglion cell layer, INL inner nuclear layer, PRL photoreceptor layer 
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Fig. 12.2 Epiretinal stimulation elicits multiple phases of spiking. The response of a RGC before 
and after administration of 40 nM CNQX (a glutamate antagonist). The early phase response in 
each trace is indicated by a black dot. The late phase response (top trace) consists of two bursts 
separated by -30 ms which was eliminated by CNQX (bottom trace). The deflection preceding the 
early phase response in both traces is an artifact of the electrical stimulus. The cell was stimulated 
with 2|aA (top trace) and 6uA (bottom trace); pulse durations were 1ms. Reprinted from [24], 
Fig. 1, with permission 



On the other hand, long duration pulses, typically those >1 ms, elicit two phases of 
spiking (Fig. 1 2.2): the first phase consists of a single action potential that occurs shortly 
after the onset of the pulse [1,9, 49], identical to the single action potential elicited by 
short duration pulses. The second phase consists of one or more bursts of action 
potentials [1,9, 12, 24, 28] and does not begin until completion of the stimulus pulse 
[9]. Responses that contain multiple bursts can last tens [9, 23] or even hundreds of 
milliseconds [1, 53]. The second phase of spiking is completely eliminated by applying 
antagonists of glutamatergic receptors, e.g. 6-cyano-7-nitroquinoxaline-2,3-dione 
(CNQX) and 2-3-dioxo-6-nitro- 1 ,2,3,4-tetrohydrobenzo[f]quinoxaline-7-sulfonamide 
(NBQX) [9, 24, 49] or by applying blockers of all synaptic activity, e.g. Cd 2+ [28]. 
These results indicate that the second phase is mediated by glutamatergic input, most 
likely resulting from the activation of bipolar cells. 

The use of whole cell patch clamp recordings allows for direct measurement of 
the synaptic input currents to RGCs and provides further support that long duration 
pulses result in excitatory input to RGCs [9, 28]. The bipolar origin of these currents 
was confirmed by Margalit and Thoreson [28] who showed that excitatory currents 
were eliminated in the presence of the glutamatergic blockers NBQX and (+)-5-mefhyl- 
10,ll-dihydro-5H-dibenzo[a,d] cyclohepten-5,10-imine maleate (MK801). Fried 
et al. [9] showed that increasing the duration of the stimulus pulse (from 1 to 3 ms) 
resulted in larger excitatory inputs suggesting that the level of bipolar cell activation 
is proportional to the total amount of delivered current and/or charge. 

Long duration pulses also elicit inhibitory activity in RGCs [9, 28] most likely as 
the result of amacrine cell activation (amacrine cells are believed to be the only source 
of inhibitory input to RGCs). There are two mechanisms however by which amacrine 
cells can become activated (Fig. 12.3). The first possibility is that long duration pulses 
activate amacrine cells directly. The second potential mechanism is that amacrine cells 
are activated secondary to the activation of bipolar cells (in the normal retina, amacrine 
cells are activated by glutamatergic input from bipolar cells). To distinguish between 
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Fig. 12.3 Long pulses activate 
amacrine cells secondary to 
bipolar cell activation. Left: 
under control conditions, long 
pulses elicit both excitatory 
and inhibitory input in RGCs. 
Inhibitory input to RGCs can 
arise either because amacrine 
cells are activated directly by 
long pulses, or as a result of 
excitatory input from activated 
bipolar cells. Right: application 
of glutamatergic blockers 
eliminated inhibitory input to 
RGCs. This indicates that 
amacrine cells are activated 
as a result of glutamatergic 
input from activated 
bipolar cells 
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the two possibilities, Margalit and Thoreson applied glutamatergic antagonists (NBQX 
and MK801) in order to block the bipolar to amacrine cell pathway. Application of 
these blockers eliminated the inhibitory input to RGCs (Fig. 12.3, right); suggesting 
that the inhibitory signal to RGCs arises secondary to activation of bipolar cells. 



12.2.1.2 The Site of Spike Initiation in RGCs 



It is important to understand which element of the RGC (soma, axon, hillock, distal 
axon, etc.) is the site of spike initiation, e.g. which element has the lowest threshold 
for spike initiation. If spikes are initiated in the soma or a nearby element, e.g. the 
axon hillock, then the neurons activated by a given stimulus pulse will be restricted 
to those that are near the stimulating electrode. This is likely to be essential for 
creating small, focal percepts. If, on the other hand, the axon has the lowest threshold, 
then elicited neural activity is not spatially restricted and the ability to create 
predictable percepts is significantly impeded. 

Several studies have attempted to identify the RGC element with the lowest 
threshold. Three different modeling studies [13, 40, 47] each concluded a different 
region (soma, axon bend, axon initial segment, respectively) had the lowest threshold. 
In physiological experiments, Jensen et al. [23] found that thresholds were lowest 
when the stimulating electrode was positioned close to the soma although the 
specific anatomical site could not be identified with their methods. Sekirnjak et al. 
[50] inferred the location of somas and axon trajectories from multielectrode array 
recordings and calculated that the site of lowest threshold was probably on the 
proximal axon, -13 urn from the soma. They too, were not able to correlate the low 
threshold region to a specific anatomical landmark. Grumet et al. [15] used multielec- 
trode array recordings to infer that spikes were initiated in the axon (based on the 
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Fig. 12.4 The band of dense 
sodium channels is centered 
in the region of low threshold, 
(a) Threshold map of a DS 
ganglion cell. The map has 
been rotated -20° in order to 
align it to the Ankyrin G 
image in (c). Dark (cool) 
areas are the lowest threshold, 
light (warm) areas are the 
highest. Thresholds range 
from 10 to 60 uA. (b) 
Overlay of the threshold map 
with staining for Ankyrin G, a 
structural protein associated 
with dense bands of sodium 
channels. The soma and axon 
(process extending leftwards 
from soma) are clearly visible. 
The white vertical lines, 
which extend up from (c), are 
used to indicate the region of 
high density Ankyrin G 
staining, (c) Ankyrin G staining 
in the same cell as in (a), (d) 
Low thresholds are also found 
in segments of the distal axon. 
The inset shows a threshold 
map that extends out along 
the distal axon, approximately 
1 . 1 mm from the soma. The 
gaps arise from incomplete 
sampling along the distal 
axon. Similar to (a), each pixel 
of the threshold map contains 
an individual threshold 
measurement; the reduced 
scale makes it difficult to 
resolve individual pixels. 
Threshold at each location 
along the axon (dashed line) 
is plotted as a function of 
distance from the soma. The 
circled point indicates the 
threshold level when the 
electrode was over the soma. 
Scale bars: 25 \aa for (a-c); 
100 irm for the inset in (d) 
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shape of the elicited spike waveform). Studies outside the retina suggest that either 
the initial segment or the nodes of Ranvier can be the target of stimulation [56]. 

To determine the site of lowest threshold, Fried et al. [10] measured threshold as 
a function of the position of the stimulating electrode. Measurements were made in 
a dense spatial grid around the soma, proximal axon and distal axon of directionally 
selective (DS) ganglion cells (one of the rabbit ganglion cell types). They found 
that thresholds were lowest in a region that was centered approximately 40 urn from 
the soma (Fig. 12.4a). Immunochemical staining revealed that a dense band of 
voltage-gated sodium channels in the proximal axon was centered at the same 
approximate location (Fig. 12.4c). Overlay of the two images reveals that the band 
of sodium channels was centered within the region of low threshold (Fig. 12.4b) 
suggesting that the band may be the source of low thresholds and possibly the site of 
spike initiation. Threshold maps were found to be qualitatively, but not quantita- 
tively similar in other cell types. 

Additional unpublished measurements from Fried et al. revealed that threshold 
levels in the distal axon were comparable to (and sometimes lower) than those of 
the proximal axon (Fig. 12.4d). These results are consistent with the Jensen et al. 
[23] physiological study as well as with the modeling studies mentioned earlier 
[13, 47]; all of which found that axonal thresholds were only slightly higher than 
the lowest thresholds (found in other portions of the cell). 

The low activation thresholds associated with RGC axons suggests that focal 
percepts may be difficult to obtain via stimulation schemes that target RGCs. This 
further suggests that stimulation methods that avoid activation of axons are needed 
in order to create spatially relevant patterns of retinal activity. 



12.2.1.3 Threshold vs. Stimulating Electrode Diameter 

The size of the electrode used to elicit activity will ultimately determine the maximum 
electrode density within the array. Therefore, small diameter electrodes presumably 
offer the highest possible spatial resolution. To explore the effects of electrode size, 
Sekirnjak et al. [49] measured threshold as a function of stimulating electrode diameter 
and found that both the current and the charge needed to elicit activity was reduced as 
the stimulating electrode diameter was decreased (Fig. 12.5a, b). However, they also 
found that both current and charge densities increased as the stimulating electrode 
diameter decreased (Fig. 12.5c, d). This trade-off arises because the reduction of 
charge associated with a smaller electrode is less than the corresponding reduction in 
electrode surface area. 

The findings from Sekirnjak et al. are supported by an earlier study from Jensen 
et al. [25], in which a 125 urn diameter electrode exhibited a reduction in threshold 
when compared to a 500 |im diameter electrode. The electrodes used by Jensen 
et al. are larger than those used by Sekirnjak et al. but support the notion that 
smaller diameter electrodes are associated with lower thresholds (charge and current). 
Whereas the above studies compare thresholds for direct activation, Ahuja et al. [1] 
found that 10 urn electrodes had higher thresholds than 200 |im electrodes when 
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Fig. 12.5 Threshold vs. electrode size. Thresholds in response to 0.1 -ms pulses (filled circles) and 
0.05-ms pulses (open circles) were plotted as a function of electrode diameter for rodent RGCs. 
(a) Current, (b) charge, (c) current density, (d) charge density. Each is plotted for the same set of 
data. All responses were long-latency spikes. Reprinted from [49], Fig. 10, with permission 



indirectly activating RGCs (via activation of presynaptic neurons). Ahuja et al. 
suggest that this may arise because the electric fields of large electrodes extend 
deeper into the retina and therefore can more easily activate presynaptic neurons. 
Further research is needed to confirm this hypothesis and also to determine whether 
other mechanisms are at work. 

Sekirnjak et al. also compared the effects of varying the pulse duration and 
found that threshold increased as the pulse duration decreased from 0.1 to 0.05 ms. 
This finding is in agreement with an earlier study by Jensen et al. [25] which found 
that thresholds increased consistently as pulse duration was reduced from 50 to 
0.1ms. These studies suggest that the smallest diameter electrode that could be 
safely used is also a function of pulse duration and therefore, short duration pulses, 
which exclusively activate RGCs, may require large-diameter electrodes. 



12.2.1.4 Spatial Extent of Activation 

Another consideration for creating focal percepts is the spatial extent of activation 
arising from a single stimulating electrode. Ideally, activation should be limited to 
the immediate vicinity of the electrode. However, the relationship between the 
strength of the electric field and the extent of activation is not well known. Several 
studies, employing a variety of methods, have begun to explore this question. 

Jensen et al. [23] measured threshold as a function of the distance between a very 
small stimulating electrode and the targeted cell body (Fig. 12.6a). They found that 
threshold was lowest when the stimulating electrode was at (or near) the soma 
(0.5 uA or 0.31mC/cm 2 ). Threshold increased as the stimulating electrode was 
moved away from the soma increasing by a factor of 20-30 at a distance of 100 urn. 
Sekirnjak et al. and Ahuja et al. [1] found similar increases in mouse and salamander 
RGCs respectively; Ahuja et al. showed that threshold increased with increasing 
distance regardless of the pulse duration (durations ranged from 60 to 1,000 |is). 
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Further support comes from a study by Schanze et al. [46] who found that 
moving the epiretinal electrode off the retina by 50 urn resulted in a 50% decrease 
in the cortical response. These results are similar to studies that found the cortical 
signal decreased when the distance between the stimulating electrode and the retina 
increased [44-46, 48]. 

The results in the retina are consistent with a large number of previous non-retinal 
stimulation studies (see [56] for a review). In general, thresholds increase with the 
square of the distance (between stimulating electrode and targeted neuron). The 
equation I = K-x -r 2 can be used to describe the increase - where I is the threshold 
current, r is the distance between electrode and neuron and K is the excitability 
constant. In non-retinal neurons, the experimentally determined excitability con- 
stant was found to be small for large, myelinated neurons and large for small, 
unmyelinated neurons. 

Jensen et al. [23] determined that the threshold for activating brisk-transient 
rabbit RGCs increased approximately with the square of the distance between 
stimulating electrode and targeted neuron (the actual threshold increase was in 
proportion to the distance raised to the 1.8 power). Jensen et al. also found that the 
rate of threshold increase was lower when the stimulating electrode was within 50 urn 
of the soma and higher for distances greater than 50 urn. While the reason for the 
different rates of increase is not known, it is possible that they arise from the long 
sodium channel bands described by Fried et al. (Fig. 12.4c). Thresholds are lowest 
when the stimulating electrode is centered directly above the sodium channel band 
and increase slowly as the electrode moves away from the center of the band but 
remains above a portion of the band. Once the stimulating electrode moves beyond 
the edges of the sodium channel band, threshold increases rapidly with increasing 
distance. Further studies are needed to confirm whether this is in fact the case. 

To get a practical sense of how the increase in threshold with distance affected the 
activation of RGCs, Sekirnjak et al. [49] used a multielectrode array that was capable 
of stimulating and recording from many closely spaced electrodes. They found that 
threshold was always lowest when the same electrode that was used to record also 
delivered the stimulus pulse; if an adjacent electrode (60|im spacing) was used to 
stimulate, threshold increased by a factor of three. This suggested that the use of low 
amplitude pulses would activate only those cells that were close to the electrode. 

To confirm that low amplitude stimulation from each electrode operated inde- 
pendently, Sekirnjak et al. independently delivered stimulus pulses from each of 
seven nearby electrodes; seven distinct responses were recorded (Fig. 12.7a, spacing 
between neighboring electrodes: 60 urn). They then activated all seven electrodes 
simultaneously and measured the response in each electrode (Fig. 12.7b). They 
found that the response elicited by activation from all seven electrodes was nearly 
identical to the response elicited by activation from a single electrode. This suggests 
that the activation from one electrode did not interfere with that of neighboring 
electrodes. This is an encouraging result as it suggests that nearby electrodes can 
independently create activity in focal regions. At stronger stimulation levels however, 
Sekirnjak et al. found that stimulation from one electrode activated RGCs in the 
vicinity of neighboring electrodes. In some cases, activity could be detected 150 urn 
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Fig. 12.7 Multiple site 
stimulation. Rat retina was 
stimulated by 7 electrodes 
simultaneously (0.8 \xA cath- 
odal pulses), (a) Overlay of 
several trials is shown for 
each electrode (1-7) and 
evoked long-latency spikes 
marked with an asterisk. 
Inset (top right): location of 
active electrodes on the array. 
Latencies ranged from 5 to 
18 ms. (b) Traces from 
neighboring electrodes 1 
(left) and 2 (right). For com- 
parison, spikes are shown for 
individual stimulation at only 
that electrode (single) as well 
as when all 7 electrodes were 
active (all). Evoked spikes 
showed no difference. 
Arrowhead indicates that the 
large spikes seen on electrode 
2 were visible on electrode 1 
as small deflections. 
Reprinted from [49], Fig. 9, 
with permission 
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from the site of stimulation. This indicates that the radius of activation is a factor 
of the stimulus strength. 

Several research groups have found that the electric field created by one elec- 
trode can interact with the field created by a neighboring electrode. Sekirnjak et al. 
found that simultaneous activation of several neighboring electrodes resulted in 
higher thresholds than that from a single neighboring electrode. Interestingly, 
these interactions were not enough to interfere with the field when the closest 
electrode was activated (Fig. 12.7b). Similarly, Ahuja et al. [1] measured thresholds 
for activating salamander RGCs from two 200 urn stimulating electrodes each 
positioned approximately 250 urn from the cell (center to center spacing). 
Thresholds for single electrode activation were approximately 13.3 nC and 
increased to 29.4 nC when stimulation from both electrodes was applied simulta- 
neously. A finite element model presented in the Ahuja et al. study indicates that 
the threshold increase arises from a reduction in the voltage gradient caused by 
simultaneous stimulation from the second electrode. 

More work is needed to determine under which conditions electrode interactions 
occur and whether there are means to reduce these interactions. One possible means 
would be to interleave the stimulus pulses from nearby electrodes - the slight offset 
in time would presumably minimize the amount of interaction between neighboring 
electrodes. 



12.2.1.5 Selective Activation 

In the normal retina, the neural activity in neighboring neurons can be quite different 
e.g. response duration of a "sustained" cell can last several hundred milliseconds 
longer than that of a "transient" cell. Similarly, ON and OFF cells typically do not 
generate spikes at the same time. This wide array of spatially and temporally varying 
neural activity is transmitted from the retina and reassembled by the visual cortex 
into our percept of the visual world. The concern arises that if prosthetic stimulation 
creates identical (or similar) activity in all neighboring neurons, the signal that 
arrives at the cortex is quite different from the normal signal and may not be intel- 
ligible. Methods to selectively activate specific RGC types may help to more 
closely re-create the signaling patterns created normally by the retina and ulti- 
mately improve the quality of the resulting percept. 

A formal study of selective activation methods for RGCs has not been 
reported. Fried et al. [10] measured thresholds in three different types of rabbit 
RGCs and found that alpha cells (Gl 1) had the lowest threshold while local edge 
detectors (LED, Gl) had the highest (Fig. 12.8). This finding suggests that low 
amplitude stimulus pulses may be able to selectively activate a single type of RGC 
(e.g. alpha). Unfortunately, this method of selective activation would at best, apply 
to a single RGC type only and would not distinguish between ON and OFF cells. 

In contrast to the results from Fried et al., Margalit and Thoreson [28] found no 
difference in thresholds between ON, OFF and ON-OFF RGCs in salamander retina. 
However, it is not clear whether the populations reported by Margalit and Thoreson 
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Fig. 12.8 Different ganglion 
cell types have different 
thresholds. Each point ("X") 
represents a threshold mea- 
surement in a different cell. 
Ganglion cell types were 
identified by the cell's light 
response prior to measure- 
ment of threshold. Pulses 
were 0. 1 ms duration, cath- 
odal with a distant ground 
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can be correlated to those from Fried et al. For example, Fried et al. found that 
thresholds for ON and OFF alpha cells were similar. In the Margalit and Thoreson 
study, it is likely that the ON-OFF cells are a different population from either the 
ON or OFF types and yet their thresholds were not different. Unfortunately, the 
limited results from Fried et al. do not preclude the possibility that some RGC types 
have similar thresholds. Further studies are needed to determine the threshold dif- 
ferences across types and if differences exist, determine whether they can be used 
to selectively activate specific RGC types. 

It is a daunting challenge to think in terms of replicating normal light elicited 
patterns with a retinal prosthesis. However, there are many incremental improve- 
ments that can be realized along the way. For example, ON and OFF varieties of 
midget and parasol cells are the four principal types of RGCs in the human retina. 
Together, it is estimated that they account for >90% of all RGCs. Methods that 
selectively activate only one of these types, for example, would likely lead to elic- 
ited patterns of neural activity that are more physiological and therefore result in 
improved percepts. 



12.2.1.6 Temporal Response Properties 

The rates at which RGCs generate action potentials [2, 6] as well as the precise 
timing with which individual action potentials [30, 31] are generated are both 
thought to play an important role in the neural code transmitted from the retina. The 
upper limit on RGC spike rates can be estimated from studies by O'Brien et al. [35] 
and DeVries and Baylor [6]. The maximum spike rates vary for each RGC type; 
alpha cells have the largest maximum spike frequency (-250 Hz) which sets an 
approximate upper limit for the response requirements of the prosthetic. 



242 



S.I. Fried and R.J. Jensen 




Fig. 12.9 Programmed sequences of short electrical pulse replicate light responses, (a) Spiking 
response to a 1-s light stimulus (horizontal bar), (b) Bottom: expanded time scale from (a) reveals 
individual spike latencies. Top: programmed sequence of short pulses derived from individual 
spike latencies: each cathodal pulse is arranged 0.5 ms before corresponding spike, (c) Spikes 
elicited by programmed sequence of short pulses (bottom) precisely match the light elicited spike 
pattern (top) from (b). Reprinted from [9], Fig. 7, with permission 



As discussed in Sect. 12.2.1.1, short duration stimulus pulses (typically 100 u.s) 
were shown to activate RGCs directly, without activating other elements of the 
presynaptic circuitry. Each short pulse elicits a single action potential [9, 49], typi- 
cally within 0.5-1. 0ms of the pulse onset [1, 9, 24, 49]. At higher stimulation fre- 
quencies, short pulses continue to elicit one spike per pulse. This was tested 
originally up to 250 Hz in rabbit [9] and more recently up to 500 Hz in salamander 
[1]. These spike rates are comparable to the fastest rates of normal, light elicited 
spiking. Using the one spike per pulse paradigm, Fried et al. programmed sequences 
of pulses in order to precisely replicate typical RGC light responses (Fig. 12.9). 

In contrast to the results from Fried et al., Sekirnjak et al. [49] found that repetitive 
stimulation at high frequencies resulted in a loss of the one spike per pulse response. 
At 50 Hz, a slight reduction (-20%) was found and a more significant reduction 
(-50%) was found at 100 Hz. It is not clear whether and/or how their findings 
impact the ability to precisely replicate light responses. 

Even if the temporal properties of normal RGC signaling can be replicated using 
short pulses, several important obstacles must be surmounted before this paradigm 
can be implemented. For example, this method would presumably activate all 
RGCs close to the stimulating electrode with the same spike patterns resulting in 
patterns of retinal activation that are non-physiological. Methods for selective acti- 
vation and avoiding the activation of passing axons are both needed. 

The temporal response properties resulting from stimulation of bipolar cells were 
very different from the responses arising from stimulation of RGCs. Fried et al. [9] 
showed that bipolar cell input to RGCs decreased as stimulus pulse frequency 
increased; by 10 Hz the amplitude of the bipolar cell output was barely detectable. 
Ahuja et al. [1] similarly found that the RGC output was almost completely eliminated 
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at 10 Hz. These findings suggest that temporal frequencies may be limited to <5 Hz if 
bipolar cells are activated (from either sub- or epiretinal stimulation). 

Since activation of bipolar cells leads to activation of amacrine cells [28], the 
reduction in bipolar cell activity may arise from amacrine cell mediated feedback 
inhibition. Therefore, methods that reduce or eliminate the secondary activation of 
amacrine cells are likely to enhance the temporal response to stimulation. Such 
methods have yet to be developed. 



12.2.2 Subretinal Stimulation 

12.2.2.1 Target of Stimulation 

Similar to epiretinal stimulation, subretinal stimulation activates many different 
classes of retinal neurons. Stett et al. [53] used pharmacological blockade of 
synaptic pathways to explore which classes of (chicken) retinal neurons were 
activated by subretinal stimulation. Under control conditions, 0.5 ms monophasic 
voltage pulses elicited RGC spiking responses whose durations lasted up to sev- 
eral hundred milliseconds (Fig. 12.10). Addition of magnesium (Mg 2+ ), a general 
blocker of neurotransmitter release, significantly reduced the RGC responses. 
The Mg 2+ sensitive spiking activity in RGCs presumably results from activation 
of one or more presynaptic excitatory neurons; the likely candidates are photore- 
ceptors, bipolar cells and/or starburst amacrine cells. To identify the specific 
neurons that were activated, synaptic blockers of the excitatory neurotransmitter 
glutamate were administered. Application of kynurenic acid, an AMPA/kainate 
receptor antagonist, greatly reduced the RGC responses to electrical stimulation. 
Kynurenic acid targets receptors at multiple locations [14, 57] but notably it 
blocks the output of bipolar cells. A more specific glutamate receptor blocker, 
2-amino-4-phosphonobutyric acid (AP4), blocks the synapse between photore- 
ceptors and ON bipolar cells [51]. Application of AP4 abolished the electrically 
evoked responses. Although Stett et al. [53] did not identify the physiological 
type of RGC shown in Fig. 12.10, the fact that AP4 abolished the evoked response 
suggests that this RGC cell was an ON cell. Jensen et al. [24] reported in a later 
study (using epiretinal stimulation) that electrically evoked responses of ON 
RGCs but not OFF RGCs in rabbit retina were abolished with AP4. 

The AP4 results suggest that photoreceptors are the principal target of electrical 
stimulation in normal retina. Understanding whether photoreceptors or bipolar cells are 
the target of subretinal stimulation has important implications for clinical use since 
patients that have been blind for many years will have few or no viable photoreceptors 
remaining (Chap. 3). Therefore methods that target photoreceptors are not likely to 
be useful in clinical applications. More research is needed to determine the relative 
excitability between photoreceptors and bipolar cells. 
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Fig. 12.10 Synaptic blockers reduce the response to subretinal stimulation. Response histograms 
(5 ms bin width) elicited by 20 repetitions of a single voltage pulses (2 V, 0.5 ms). Application of 
high [Mg 2+ ], kynurenic acid or AP4 reduced the spiking activity. The number at each histogram 
indicates the time interval (minutes) after switching to the perfusate with the agents given at the 
right and to the standard perfusate for washing out the agents. Scale bars 100ms, lOOspikes/s. 
Reprinted from [53], Fig. 6, with permission 

12.2.2.2 Threshold vs. Polarity of Stimulation Pulse 



It is well known that axons (including those of RGCs) are more sensitive to a cathodal 
current pulse than to an anodal current pulse [56]. When RGCs are activated through 
electrical stimulation of presynaptic cells, the situation is not so straightforward. 

Jensen and Rizzo [19] reported that when the rabbit retina is stimulated with a 
subretinal electrode the threshold current needed to activate OFF RGCs was much 
lower for an anodal current pulse than for a cathodal current pulse. On the other 
hand, cathodal and anodal current pulses were on average equally effective for 
activating ON RGC cells. This is illustrated in Fig. 12.1 1, in which threshold mea- 
surements were made for RGC responses to stimulation of the neural network. 
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Fig. 12.11 Threshold charge as a function of stimulus pulse duration for OFF ganglion cells (left 
graph) and ON ganglion cells (right graph) for both cathodal and anodal stimulus pulses. For OFF 
ganglion cells (left), anodal stimulation produces a substantially lower activation threshold for all 
pulse durations, while as a whole ON ganglion cells (right) are insensitive to the polarity of 
stimulation. Reprinted from [19], Figs. 4 and 5, with permission 



In the chicken retina, Stett et al. [54] reported that when the neural network is stimulated 
with a subretinal electrode, anodal voltage pulses were overall more effective than 
cathodal voltage pulses. They found that on average a 3.2-fold difference in thresh- 
olds. They did not distinguish between ON and OFF RGCs. Nevertheless, both 
studies suggest that for indirect activation of RGCs (with a subretinal electrode) an 
anodal stimulus is in general more effective than a cathodal stimulus. The findings 
of Jensen and Rizzo [19] further suggest that a cathodal current pulse may bias 
activation of ON cells over OFF cells. Results such as these may one day underlie 
methods to selectively activate ON vs. OFF pathways which would allow more 
physiological patterns of activity to be elicited. 

In contrast to the ON-OFF selectivity found in rabbit (described above), a recent 
study conducted in the mouse retina [22] found that the median threshold current 
for cathodal stimulation of ON RGCs was only 32% lower than for OFF RGCs and 
this difference was not statistically significant. Thus, it would seem from the mouse 
experiments that a cathodal current pulse may not bias activation of ON cells over 
OFF cells as the findings in the rabbit would suggest. It will be of interest to examine 
the thresholds of ON and OFF RGCs to anodal and cathodal current pulses in the 
primate retina. 



12.2.2.3 Spatial Extent of Activation 

Stett et al. [54] examined the spatial extent of activation of RGCs in the chicken retina. 
They used an ultra- fine (1-u.m diameter) tip electrode for stimulating the retina and 
a dense multielectrode array to record simultaneously from many RGCs in the retina. 
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They reported a half width of an "electrical point spread function" of -100 urn. This 
distance on the retina corresponds to a visual angle of 21' in the human eye. 
A minimum angle of resolution of 21' corresponds to a visual acuity of -20/400. It 
will be of interest to examine the electrical point spread function of RGCs in the 
primate fovea where the convergence of photoreceptors and bipolar cells onto RGCs 
is very low. The findings may indicate that a higher visual acuity is possible. 



12.2.2.4 Temporal Response Properties 

Fried et al. [8] showed that when rabbit RGCs are indirectly activated with an epireti- 
nal stimulating electrode, bipolar cell output is drastically reduced by a 10Hz stimula- 
tion frequency. The situation is not much different with a subretinal stimulating 
electrode. Jensen and Rizzo [20] showed that the responses of rabbit RGCs to stimu- 
lation of the neural network began to diminish in size when the retina was stimulated 
within ~400ms of a preceding current pulse (Fig. 12.12). The shorter the interpulse 
interval, the smaller was the response to the second stimulation pulse. They also 
studied the responses of RGCs to trains of pulses applied at different frequencies. As 
expected, the responses were greatly reduced for stimulation frequencies >25Hz. 
These data indicate that rapid electrical stimulation of the retina in patients with a 
retinal prosthesis may be counterproductive, assuming that RGCs are being activated 
through the neural network. 



Fig. 12.12 Mean paired- 
pulse depression of RGC cell 
response amplitudes in rabbit 
retina. Data were collected 
using biphasic current pulses 
of 1 ms per phase. Ampl 
amplitude of first response; 
Amp2 amplitude of second 
response. Reprinted from 
[20], Fig. 2, with permission 
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Fig. 12.13 The average number of spikes elicited in two RGCs (a, b) as a function of stimulus 
charge. The responses of both cells increased with increased charge injection. While the response 
of the cell on the left (a) appeared to plateau with high charge injection, the response of the cell 
on the right (b) fell with high charge injection. Both cells were more sensitive to an anodal pulse. 
Reprinted from [54], Fig. 3, with permission 



12.2.2.5 Dynamics of the Retinal Response 

Stett et al. [53] found that the number of spikes in an evoked retinal response increased 
with increasing voltage level. In a later study [54], they reported that the number of spikes 
evoked per voltage pulse was almost a logarithmic function of the charge delivered 
(Fig. 12.13). This finding suggests that it may be possible to influence the intensity of a 
visual percept in a patient with a retinal prosthesis by adjusting the amplitude of the cur- 
rent pulses. Unfortunately, further increases in the injected charge eventually led to a 
decrease in the number of spikes for some cells (Fig. 12.13b). 



12.2.2.6 Comparing Sub- vs. Epiretinal Stimulation 

It is difficult to draw definitive conclusions from the existing studies that compare the 
thresholds elicited by sub- vs. epiretinal stimulation. O'Hearn et al. [36] measured 
thresholds in the mouse retina using two 125 urn disk electrodes (bipolar configuration) 
and 1 ms duration biphasic pulses (cathodal first). Thresholds for epiretinal stimulation 
were 30 uA (0.61mC/cm 2 ) while thresholds for subretinal stimulation were 77 uA 
(1.57mC/cm 2 ) suggesting that epiretinal stimulation is more effective (Table 12.1). 
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Table 12.1 Comparison of sub- vs. epiretinal thresholds 



O'Hearn et al. 


(125 um diameter electrode, biph 


asic pulses) 


Pulse duration 


Epiretinal 


Subretinal 




1.0 ms 


30 uA 


77 uA 




Jensen et al. (500 um diameter electrode) 




I 


II 


III 




Epiretinal 


Subretinal 


Subretinal 


Pulse duration 


Cathodal 


Cathodal 


Anodal 


0.1ms 


54 uA 


180 


29 


2.0 ms 


6.3 uA 


6.7 


1.3 



The results of two different Jensen et al. studies [19, 25] can also be used to 
compare thresholds. Both studies used identical stimulation parameters, including a 
500 urn stimulating electrode and pulse durations of 0.1 and 2.0 ms and were 
restricted to OFF RGCs of the rabbit retina. The thresholds in response to epiretinal 
stimulation were lower than those from subretinal stimulation for both pulse durations 
tested (0.1 and 2.0ms); however the difference was small for 2.0ms pulses. While 
these results are qualitatively in agreement with O'Hearn et al., Jensen and Rizzo point 
out that it may be more accurate to compare epiretinal cathodal pulses to subretinal 
anodal pulses since in both cases current flows through the retina in the same direction 
[19]. Under these conditions, subretinal thresholds are significantly lower with both 
short and long pulses (Table 12.1, bottom, compare columns I and III). 

It is difficult to assess the discrepancies between the O'Hearn et al. and Jensen 
and Rizzo studies. Both the stimulus waveform and electrode size are different 
between studies and either (or both) may contribute to threshold differences. In 
addition, the O'Hearn study was unable to distinguish between the early and late 
phase responses and therefore comparison with the Jensen et al. studies may not be 
appropriate. In addition, neither group was able to ascertain whether photoreceptors 
were activated by stimulation raising the possibility that the differences arise 
because different neurons were activated in each study. Further research is needed 
to better understand the relative thresholds and underlying mechanism differences 
between sub- and epiretinal stimulation. 



12.3 Electrophysiological Properties of RGCs 
in Degenerate Retina 

Electrophysiological studies on RGCs in degenerate retina have been made in the 
rdl mouse and the dystrophic Royal College of Surgeons (RCS) rat. Similar to a 
form of retinitis pigmentosa (RP) in humans, the rdl mouse has a mutation in the 
gene for the P-subunit of cGMP phosphodiesterase-6 [7]. As a consequence, rapid 
rod photoreceptor degeneration begins at approximately postnatal day (P)10, with 
nearly all photoreceptors lost by P36 [3]. The RCS rat has a mutation in the receptor 
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tyrosine kinase gene Mertk, which results in the failure of the outer segments of 
photoreceptors to be phagocytosed and eventually causing the death of photoreceptors 
[5, 34]. Degeneration of photoreceptors in RCS rat is slower than that in rdl mouse. 
Only -1/3 of photoreceptors are lost by P30. However, by P75 only scattered pho- 
toreceptors are present [37]. 

Pu et al. [38] reported that RGCs in RCS rats show an increased level of baseline 
spiking and by P47 (before complete degeneration of photoreceptors) were 
predominately OFF cells. Similar findings were reported in the rdl mouse retina [52] 
where baseline spiking levels went from <1 Hz in RGCs of normal, wild type mice to 
as high as 20 Hz in retinas from rdl mice. Furthermore, Stasheff [52] found that -2/3 
of rdl RGCs exhibited rhythmic bursts (peaks at -6 and -12 Hz) of activity. As in the 
RCS rat retina, Stasheff [52] reported that ON and OFF responses are differentially 
affected in early stages of retinal degeneration rdl mouse retina. Light-evoked ON 
responses in rdl RGCs at P14-P15 were reduced more than OFF responses, and 
unlike OFF responses many of the ON responses showed an increased latency. 

From patch-clamp recordings, Margolis et al. [29] reported rhythmic spike activity 
(frequency of -10 Hz) in rdl mouse retinas of P36-P50. The spike bursting was 
present in older mice as well, although the frequency of the bursts decreased 2-3-fold. 
Margolis et al. further showed that the rhythmic bursting was not generated intrinsi- 
cally in RGCs but was due to strong, aberrant synaptic input. Intrinsic electrical 
properties of (morphologically identified) ON and OFF RGCs in rdl mouse retinas 
were similar to those in wild-type mouse retinas. 

The hyperactivity and bursting activity in RGCs indicates that new strategies for 
prosthetic stimulation of the retina may be necessary. For example, higher spiking 
levels may be needed to obtain an adequate signal to noise ratio. Alternatively, it 
may be necessary to develop stimulation strategies that reduce the level of baseline 
activity in order for the brain to be able to understand the signal leaving the retina. 
However, it is important to first determine whether increased spiking levels and 
bursts of activity are present clinically (in patients) before new stimulation strate- 
gies are developed. 



12.4 Responses of RGCs to Electrical Stimulation 
in Degenerate Retina 

The prosthetic must ultimately function in patients with retinal degenerative disease 
and therefore stimulation methods for clinical use must be tailored to these types of 
retinas. While many studies have examined the structural changes that occur as part 
of the degenerative process (Chap. 3), the corresponding physiological changes are 
not as well known. It is important to understand how the anatomical and physiolog- 
ical changes in degenerate retina will affect the response to electrical stimulation. 
In this section, we explore the few studies that have looked at the electrophysiological 
properties of RGCs in degenerate retina as well as those that have looked at the 
responses of RGCs in these retinas to electrical stimulation. 
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12.4.1 Epiretinal Stimulation 

Suzuki et al. [55] measured the thresholds of RGCs in wild-type and 16-week old 
rdl mouse retinas to biphasic current pulses delivered through a 125-|im electrode 
positioned on the epiretinal surface. The thresholds were on average 20-50% 
higher in rdl mouse retinas, depending on pulse duration. A description of the 
evoked responses was not provided so it is unclear whether the thresholds were 
from direct or indirect activation of the RGCs. 

O'Hearn et al. [36] examined the thresholds for activation of RGCs in 8-12 
week old wild-type and rdl mouse retinas. Retinas were stimulated with a pair of 
electrodes (125-|im diameter) that were positioned either epiretinally or subreti- 
nally. With epiretinal stimulation, the thresholds for activation of RGCs in rdl 
mouse retinas were 1.8-fold higher. The short-latency responses suggest that the 
RGCs were directly activated although this was not discussed by the authors. If so, 
then perhaps the properties of the sodium channel bands that presumably underlie 
RGC activation thresholds (described in Sect. 12.2.1.2) change during the degen- 
erative process and the change results in a higher threshold. 



12.4.2 Subretinal Stimulation 

12.4.2.1 Response Properties of RGCs 

Jensen and Rizzo [21] compared the electrically evoked responses of RGCs in 
wild-type and rdl mouse retinas to stimulation of the neural network. Using 
biphasic current pulses, they found that RGCs in rdl mouse retinas respond 
similarly to wild-type RGCs. In both wild-type and rdl mouse retinas, three 
types of electrically evoked responses were observed (Fig. 12.14). Type I cells 




Wild type Rd1 

Mouse strain 

Fig. 12.14 Dot plot of thresholds for activation of RGCs in wild-type and rdl mouse retinas with 
biphasic current pulses. See Sect. 12.4.2.1 for a description of the three types of RGCs. Reprinted 
from [22], Fig. 4, with permission 
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elicited a single burst of spikes within 20 ms following application of the electrical 
stimulus, type II cells elicited a single burst of spikes with a latency greater than 
37 ms, and type III cells elicited two or more bursts of spikes. The similarity of 
electrically evoked responses in wild-type and rdl mouse retinas led Jensen and 
Rizzo to suggest that postreceptoral neurons (not photoreceptors themselves) 
determine the response properties of RGCs to electrical stimuli. 



12.4.2.2 Activation Thresholds of RGCs 

As noted above (Sect. 12.4.1), O'Hearn et al. [36] examined the thresholds for 
activation of RGCs in wild-type and rdl mouse retinas with a pair of electrodes that 
were positioned either epiretinally or subretinally. With subretinal stimulation, the 
thresholds for activation of RGCs in wild-type and rdl retinas were not statistically 
different. In contrast, Jensen and Rizzo [21] reported that the thresholds of RGCs 
in rdl mouse retinas were 3.6-fold higher than the thresholds of RGCs in wild-type 
mouse retinas. The elevated thresholds occurred for all ages examined, ranging 
from postnatal day (P) 25 to PI 86. Of the three types of RGCs identified (see 
Sect. 12.4.2.1), type I RGC cells appeared to be particularly affected. 

The discrepancy in the findings between Jensen and Rizzo [21] and O'Hearn 
et al. [36] may be because Jensen and Rizzo were examining the thresholds of 
RGCs due to indirect activation (through the network), whereas O'Hearn et al 
might have been examining thresholds of RGCs to direct activation, judging from 
the short-latency (<3 ms) responses they reported in their study. 

Although Jensen and Rizzo [22] proposed that photoreceptors lower the thresholds 
for activation of RGCs, the reason for the higher thresholds in rdl mouse retinas is 
not yet fully understood. Bipolar cells provide most of the excitatory input to RGCs. 
Perhaps, a greater amount of depolarizing current is needed to stimulate neurotrans- 
mitter release from these cells, either from the absence of facilitatory effects from 
photoreceptors or due to early structural changes in the bipolar cells themselves. 



12.5 Cortical Responses to Retinal Stimulation 

12.5.1 Spatial Properties Revealed by Cortical Measurements 

Several studies have explored the cortical response that result from electrical stimula- 
tion of the retina. These types of studies not only confirm that elicited retinal activity 
is transmitted to higher visual centers, they also provide a more global view of the 
results of stimulation. The position of cortical electrodes can vary from surface (scalp) 
to penetrating: less invasive methods typically record electrically elicited cortical 
potentials (EECPs) - slow potentials that reflect activity of large populations of neu- 
rons. On the other hand, more invasive methods (e.g. penetrating electrodes) record 
brief activity (action potentials) from only a few neurons (typically one or two). 
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12.5.2 Local Field Potentials 

Penetrating electrodes that are appropriately positioned and configured can also 
record local field potentials (LFPs). Whereas spiking measures neuronal output, 
LFPs are thought to reflect input signals to the neuron and arise from the electric 
currents associated with dendritic activity of a relatively large number of nearby 
neurons [26, 27, 32, 33]. The time course of the LFP signal contains one early and 
several late components; the early component reflects direct synaptic input while 
later components reflect later processing most likely arising from intracortical 
connections [7, 59]. By carefully positioning the recording electrodes at layer IV of 
visual cortex (the input layer), and looking at only the early LFP components, a 
measure of the input signal to the visual cortex can be obtained. Presumably this is 
a direct assessment of the retinal output arising from electrical stimulation. 



12.5.3 Elicited Responses Are Focal 

Similar to the findings of Sekirnjak et al. [49], Schanze, Wilms, Eger and colleagues 
[8, 43, 45, 46, 59] found that retinal stimulation from a multielectrode array elicited 
small focal and spatially discrete regions of cortical activity (LFPs). The average 
size was approximately 1.3 mm of cortex corresponding to a visual angle of 1.5° 
although nearly 10% of the regions were much smaller (-0.4 mm cortex/0. 5° visual 
angle). Similar sizes (0.68° visual angle) were obtained by Cottaris and Elfar [4] in 
a recent study that used similar methodology. Activated regions elicited by indi- 
vidual stimulating electrode were topographically arranged [46, 59] confirming that 
electrical stimulation is transmitted retinotopically. The activated cortical regions 
and corresponding stimulating electrodes were not perfectly aligned however [46, 
59], possibly because of limited resolution from the cortical array or possibly 
because electrical stimulation activated axons of distal neurons. 

The width of the activated cortical region increased with the amplitude of the 
stimulus pulse [59]. At threshold, the width of the activated region was smallest 
(average -1°) increasing by 10% in response to doubling the amplitude and by 
67% in response to a tenfold increase in amplitude. These increases are smaller 
than might be expected since a tenfold increase in amplitude would more than 
double the retinal area containing activated neurons. In contrast to the increases 
observed by Schanze and colleagues, Cottaris and Elfar [4] observed a reduc- 
tion in width with increases in stimulus strength. Cottaris and Elfar speculate 
that the differences may arise from the differences in pulse durations used 
across the two studies, perhaps because different pulse durations target different 
classes of retinal neurons. Further study is needed to resolve the mechanism(s) 
by which patches of retinal activity are transformed into patches of cortical 
activity. 
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12.5.4 Cortical Measurements Reveal Electrode Interactions 

A series of papers by Schanze, Wilms, Eger and colleagues [8, 43, 45, 46, 59] found 
evidence of interactions between nearby electrodes. Schanze et al. [45] found that 
both excitatory and inhibitory interactions occur and that the sign of the interaction 
depended on the distance between the retinal stimulating electrodes. Specifically, 
they found that electrode separations of 3-4° generated excitatory interactions 
resulting in cortical activation regions that were larger than the sum of the two 
individual regions. Smaller or larger electrode separations generated the opposite 
effect: inhibitory interactions where the size of the activation regions was less than 
the sum of the individual regions. 

These studies are seemingly in conflict with those of Sekirnjak et al. [49] how- 
ever, there are considerable methodological differences between the studies that 
might underlie the observed differences. For example, while the Sekirnjak et al. 
study reflects the activity of a limited number of RGCs, the cortical measurements 
presumably reflect the activity of a large population of retinal neurons. In addition, 
the input activity to cortex reflects synaptic processing that occurs in the thalamus. 
Any of these factors might account for the electrode interactions observed in the 
cortical studies. Understanding these interactions will be important for effectively 
constructing complex spatial images from individual percepts (phosphenes). 



12.5.5 Temporal Responsiveness in Cortex 

The maximum spiking rate in cortical neurons is typically around 50 Hz [16, 39]; 
considerably slower than the maximum rate of 250 Hz generated by RGCs 
(Sect. 12.2.1.6). This reduction in maximum spike rate is thought to occur at the 
synapse between RGCs and lateral geniculate neurons. Therefore, it is important to 
understand how the temporal parameters of retinal stimulation translate into the 
temporal response patterns of cortical neurons. 

To study this, Wilms et al. [59] measured the temporal properties of the LFP 
early component. As mentioned earlier the early component of the LFP is thought 
provide a representative measure of the retinal output. Rise times (10-90%) ranged 
from 4.8 to 8.3 ms with lower values arising from higher amplitude stimuli. A sine 
curve was best fit to each rise time; the resultant frequencies of 40-70 Hz were 
obtained by multiplying each rise time by three. More sophisticated methods of 
estimating the temporal responsiveness yielded similar results. This suggests that 
that adequate temporal resolution may be achievable with electrical stimulation of 
the retina. Additional findings from Wilms et al. [59] suggest that larger amplitude 
pulses create better temporal resolution. 

Unfortunately, the large amplitude stimulus pulses that increase temporal resolution 
may simultaneously lower the spatial resolution. The authors suggest that in clinical 
use these parameters could be varied depending on whether the patient has a more 



254 S.I. Fried and R.J. Jensen 

pressing need for spatial or temporal information. The study from Cottaris and Elfar 
[4] however suggests that such a tradeoff may not be necessary since they did not 
see a decrease in spatial resolution; additional studies are needed to resolve this. 



12.6 Suggestions for Future Studies 

Several important research goals emerge from a review of the studies to date. First, 
the thresholds associated with eliciting activity in RGCs during animal studies are 
consistently low - well below the charge density limits considered to be safe for 
electrodes. These low levels are in contrast to the much higher thresholds required 
to elicit percepts during clinical trials [17, 18, 41]. Elevated thresholds raise many 
concerns. For example, larger power supplies that generate more heat will be 
required. Also, the diameter of the stimulating electrodes will need to be increased 
in order to maintain charge densities below established safety levels. Unfortunately, 
increasing the size of the electrode reduces the potential resolution of these devices. 

The reasons underlying the threshold differences are not well understood. 
Possibilities include structural and functional alterations in the diseased retina, vari- 
ability in the distance between electrode and neurons (smaller and more controllable 
during animal studies), uncertainty in the intactness of the ascending visual path- 
ways in patients that have been blind for many years, and appropriateness of the 
stimulation methods used to elicit clinical percepts. Studies that systematically 
elucidate which factors contribute most to threshold differences are needed. 
Presumably, an improved understanding of the factors that influence thresholds will 
lead to more efficient stimulation methods for eliciting clinical percepts. 

A second research goal that emerges is to develop better methods of stimulation. 
Existing retinal prosthetics typically use stimulating electrodes with diameters that 
are 10-20 times larger than the diameter of RGC somata. Stimulation from these 
larger electrodes presumably results in similar patterns of elicited neural activity in 
large numbers of RGCs situated in and around the electrode region. Light responses 
from neighboring RGCs (of different types) utilize different patterns of spike activity - 
differences can include variations in both spike frequency and total spike count. 
Thus, the prosthetic elicited patterns of activity are considerably different than 
normal physiological patterns. Stimulation methods that bring the elicited neural 
activity closer to physiological patterns are likely to improve the quality of elicited 
vision, even if every aspect of normal signaling patterns cannot be replicated by 
existing devices. 

The need for improved stimulation methods becomes even more necessary after 
considering the likely clinical applications for retinal prosthetics. The most common 
form of retinal blindness arises from age-related macular degeneration (AMD). 
However many patients with AMD retain some useful peripheral vision. Therefore, 
surgical implantation of the prosthetic needs to provide clinical benefit that out- 
weighs the risk of damage to existing vision. Sophisticated methods of activation 
will be to needed to achieve these high levels of vision. For patients that are 
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completely blind, the criteria for success may be lower but will still need to meet 
or exceed the information provided from a white cane or guide dog in order to 
justify the risk and the costs associated with implantation. Once again, fairly com- 
plex methods of stimulation are likely to be needed. 

The third research goal is to develop a better understanding of the changes that 
occur in the retina as part of the degenerative process. For example, studies that 
show that background spiking levels increase in RGCs of the degenerate retina sug- 
gest that the prosthetic may not only need to create spiking activity, it may also 
need to suppress activity as well. Changes in baseline activity must be fully under- 
stood before appropriate stimulation schemes can be developed. In addition, several 
genetic models of retinal degeneration exhibit drastic changes in both cell structure 
and synaptic connections. If portions of the inner retina are destroyed, stimulation 
schemes that target neurons presynaptic to RGCs may not be effective. It is necessary 
to fully understand these changes before appropriate stimulation methods can rationally 
be developed. Interestingly, since clinical trials (in blind subjects) indicate that 
large areas of the human retina remain viable and that retinotopic wiring persists, it 
is possible that the degenerative process in humans may be less severe than those 
reported in laboratory animals. 

Finally, while methods for selective activation of individual types of RGCs await 
development, they promise great insight to our understanding of visual processing. 
For example, if it were possible to selectively activate a single population of RGCs 
(e.g. midget or parasol), its role in visual perception could be explored. If more than 
one population could be independently activated, knowledge of how multiple RGC 
types act in concert could also be explored (e.g. do spikes from the two types need 
to be generated synchronously?). Similar studies could be performed to elucidate the 
roles of the ON and OFF systems - a possibility arising from work that indicates 
these two types may have different thresholds in response to subretinal stimulation. 
Questions such as these have been difficult to explore using more conventional 
research tools but offer tremendous insight into the function of the visual system 
once appropriate methods are developed. 
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Chapter 13 

Findings from Acute Retinal Stimulation 

in Blind Patients 



Peter Walter and Gemot Roessler 



Abstract In acute retinal stimulation experiments retinal stimulators are inserted 
into the eye, activated, and responses from patients to electrical stimulation are 
recorded. These tests were done to obtain evidence that the principle of electrical 
stimulation of the retina works in terms of elicitation of phosphenes or visual per- 
ception, respectively. These tests were also done to narrow the parameter range for 
electrode size and stimulation energy before efforts were undertaken to fabricate a 
device for chronic stimulation. Results from such tests were also helpful to describe 
possible perception patterns of patients and also to estimate possible visual acuities 
after implantation. Usually these tests were done in local anaesthesia so that the 
patient can respond verbally or by means of an interface to the stimulation. 
In different experiments rheobase and chronaxie data were reported showing a 
large variation depending on the device and on individual factors such as the disease 
state or the proximity between the electrode and the retina. Possible spatial and 
temporal resolution data were calculated from such experiments demonstrating that 
the concept of retinal stimulation in blind RP subjects can really help to restore 
some useful visual function in such patients. 
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RI 


Response interface 
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Retinitis pigmentosa 


SIU 


Stimulus isolation unit 
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Stimulator 
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13.1 Introduction 

The basic assumption behind the development of implantable devices for retinal 
stimulation is that electrical stimulation of the retina may provide useful vision in 
patients suffering from advanced forms of degenerative diseases of the retina. 
Either from theoretical considerations but also from early experiments in blind 
human subjects one may conclude that this assumption should be correct. An early 
example of a human experiment was the implantation of stimulation electrodes 
across the visual cortex as reported by Brindley and associates. A blind RP patient 
reported phosphenes upon electrical stimulation of the visual cortex [1]. Dobelle 
and his group continued the work of Brindley and they were also able to demon- 
strate that blind subjects do have visual sensations when the posterior parts of the 
visual system are electrically stimulated [5]. The application of electrodes onto, 
underneath, or within the retina was limited to basic research approaches and did 
not extend to therapeutic efforts. Not earlier than 1991 devices and surgical tech- 
niques became available with which in patients suffering from retinitis pigmentosa 
(RP) experiments for retinal stimulation could be performed in the operating room 
without considerable risk to the patients. The rationale to do these experiments was 
that only data was available from retinal stimulation experiments in animals with a 
normal retina using preliminary electrode arrays or in tissue preparations of RCS 
rat retina using multielectrode array devices but not implantable electrodes. From 
these animal experiments only some information was known about the range of 
stimulation currents and about the timing of the stimulation pulses. It was not 
known to what extent the stimulation parameters would have to be changed to 
achieve visual percepts in blind humans suffering from such a disease. Three major 
questions should be answered by acute retinal stimulation experiments in humans, 
(a) Is it possible to elicit visual percepts when stimulation pulses are emitted by 
electrodes placed near the degenerated retina? (b) What charge delivery is neces- 
sary to obtain such responses? (c) Is it possible to elicit several percepts when 
several electrodes are activated and what is the two-point discrimination? All three 
questions were crucial. If it was found that the energy required to obtain visual 
percepts in RP patients was above the maximum charge delivery capacity of the 
electrode material or beyond a level indicating toxic tissue reactions then it would 
not have been possible to further pursue these research projects. If only unpatterned 
chaotic percepts were registered than there would also be no chance to establish 
artificial vision in terms of useful vision. 
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13.2 General Considerations for Acute Retinal Stimulation 
Experiments 

Acute experiments for retinal stimulation in blind humans require the possibility to 
measure more or less quantitatively the visual response. Objective measurements 
are not possible in a clinical setting because the obtained local responses are too 
small to detect them with surface electrodes attached to the skull, although Chen 
reported one blind patient in which he recorded evoked cortical potentials with 
scalp electrodes upon electrical stimulation of the retina with eight electrodes 
simultaneously and 10% above threshold [3]. Due to obvious reasons microelec- 
trodes inserted in the visual cortex to record local field potentials or functional 
imaging experiments in humans were not performed in contrast to such experi- 
ments which have been reported for animal studies [9, 18]. 

Acute tests for electrical stimulation of the retina have to be performed under 
local anesthesia. Only superficial anesthesia techniques such as subconjunctival or 
subtenon injections are recommended because any effect of the anesthetic drug on 
the optic nerve must be excluded. Sedative drugs should also be avoided because the 
patient has to indicate the visual response either by voice but more reliable by a 
response interface such as a set of buttons which he is asked to press to indicate 
whether he sees something or not. All patient responses must be recorded using such 
response interfaces to correlate them afterwards with the stimulus parameters. When 
using single electrodes, stimulus thresholds can be recorded by a two- alternative 
forced choice method at several points of the retinal surface. 

When using electrode arrays, stimulus threshold data can be determined for 
each electrode or electrode pattern. Electrode arrays could also be used to esti- 
mate if two points or lines can be differentiated by the patient when two electrodes 
or two clusters of electrodes are stimulated simultaneously. Information on the 
distance of distinguishable electrodes or angles should give some information on 
the possible visual acuity that can be achieved with such systems. Important aspects 
of the neurophysiology of the target tissue can also be investigated, such as the 
determination of rheobase, which is the minimum stimulus intensity necessary to 
elicit a response at very long stimulus durations, and chronaxie which is the stimu- 
lus duration necessary to elicit a response at twice the rheobase level of stimulus 
strength. These data are characteristic for certain elements of nervous tissue. 

The main limitation of acute retinal stimulation experiments in humans is that the 
time to perform these experiments is limited. Usually 1 h of experimentation is pos- 
sible. Within this time all the possible combinations of stimulus intensity and time at 
all electrode positions cannot be included in the experimental setup. Another limita- 
tion is that the patient's response is not a uniform standardized yes or no. The answer 
sometimes also contains information on shape or color or maybe on temporal aspects 
of phosphenes. This information can usually not be interpreted systematically. 

It should also be pointed out that acute tests for retinal stimulation have been 
performed in two types of patients: blind patients with RP and patients in which the 
eye has to be removed because of cancer. In the latter the retina itself usually was 
normal [12-14, 19]. 
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13.3 Surgical Technique 

Full pupil dilation should be obtained and then the patient is prepared for vitrectomy. 
Sclerotomies are made 3-4 mm behind the limbus. A vitrectomy is performed to 
avoid any traction at the entry sites or elsewhere to the retina. Wide angle viewing 
systems are indispensable. The size of the sclerotomy depends on the size of the 
implant. Usually handheld devices are used for acute retinal stimulation experi- 
ments. These devices are held onto or above the retinal surface. They are usually 
connected via a cable with a programmable power unit providing the requested 
pulse sequences to each electrode (Fig. 13.1). The precision with which such 
devices are held to the retinal surface is usually not constant throughout the experi- 
mental procedure. In such approaches eye movements may be a problem. 
Therefore some authors suggest the use of botulinum toxin to achieve akinesia 
[14, 15]. Movement of the device should be avoided during the stimulation proce- 
dure for several obvious reasons. Threshold determination may vary significantly 
depending on the force with which an electrode is pushed towards the retinal surface 




I x t diagram 









Rheobase 



— 1+ 



Chronaxie 



Fig. 13.1 Typical 1 x t diagramme for the electric stimulation of neural tissue. The I x t diagramme 
is determined by finding the stimulus current for a given stimulus duration or by finding the 
stimulus duration for a given stimulus current necessary to evoke a certain response, usually 
the threshold response. The minimal current to evoke a response with very long stimulus durations 
is called rheobase. The stimulus duration at the twofold rheobase intensity is called chronaxie. 
Rheobase and chronaxie are values characteristic for certain tissues and stimulation settings. The data 
points are experimental data fitted by a mathematical model 
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and also depending on the location of the electrode. For animal experiments 
therefore devices were used which were placed onto the retinal surface and held 
here in place with heavy liquids such as Perfluorodekaline [17]. Rizzo and 
coworkers used gold weights to apply pressure to the devices in a series of human 
experiments [14]. 

Quantification of the precision in terms of distance between electrodes and retina 
or pressure between retina and electrode and the constancy of the position is diffi- 
cult. Even with such tools movement of the array may occur during the experiment 
as mentioned by Rizzo [15]. Therefore, the conclusions drawn from such experi- 
ments should be regarded cautiously. It is important when such experiments are 
performed and their results interpreted that the position of the array on the retinal 
surface is known. Much better information could be gained with experiments where 
the electrode array is chronically mounted onto the retina. Weiland and coworkers 
found in acute retinal stimulation tests in normal eyes that lifting an epiretinal elec- 
trode more than 0.5 mm off the retina resulted in loss of the electrically evoked 
percept [19] (Figs. 13.2 and 13.3). 

After removal of the implant the sclerotomies are closed. Clinically, the patients 
showed adverse events in rare cases only. As in every vitrectomy the patient should 
be informed that a retinal detachment may occur in up to 5% of cases, as may cata- 
ract formation or in rare cases endophthalmitis. Such adverse events may require 
secondary interventions. 




Fig. 13.2 Left; General setup for acute experiments on retinal stimulation. Under vitrectomy 
conditions the stimulator (STIM) is handheld at the desired position. A light probe (LP) is also 
inserted to allow visualization of the stimulator position onto the retinal surface. The electrodes 
are connected to a power source (PS) controlled by a computer system (PC) and possibly using a 
multiplexer (MUX) if several electrodes are desired. The electrodes are physically isolated from 
the high voltage devices using stimulus isolation units (SIU). The patient's responses are regis- 
tered using response interfaces (RI) and the whole procedure is usually video documented (VD). 
Right; Intraoperative situation during an acute experiment for retinal stimulation - surgeon's view, 
inferior retina is in the upper part of the picture. The handheld microelectrode device is placed 
onto the retinal centre with the active electrodes near the superior arcade 
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Fig. 13.3 Examples of probes for acute retinal stimulation experiments in human, (a) Polyimide 
stimulator used by Hornig et al. [10]. The thin and flexible polyimide device has a width of 
1.6 mm. Active electrodes with diameters between 50 and 360 urn are placed at the tip of the 
device, the return electrodes are at the base of the device adjacent to the metal housing, (b) 17.5 
Gauge steel cannula with several stimulation electrodes at the tip as used by Humayun and co- 
authors [12]. (c) 125 iim thin curved platinum wires for direct electrical stimulation of the retinal 
surface as used by Humayun and co-authors [12] 



13.4 Threshold Measurements 



The first approach in such settings is always the determination of thresholds at 
which a stimulation pulse or pulse series yields a phosphene. A two alternative 
forced choice method is usually applied. In such experiments threshold is com- 
monly defined as the lowest stimulus intensity at which on 75% or more of the test 
repetitions the patient correctly reports a visual percept. 

Also catch trials are usually employed in acute retinal stimulation testing, i.e. a 
stimulus is indicated, e.g., by a warning tone, but no electric pulse is given. If the 
subject gives a positive answer the answer is classified as false positive. Such catch 
trials are needed to test the reliability of a subject in this very demanding situation. 
Only those experiments should be analyzed in which patients do not give too many 
false positive responses. The criterion at which tests should not be used because of 
too many false positive responses should be defined for each experiment [10]. 

Threshold measurements were reported by Rizzo [14, 15]. In his series visual 
responses in patients with advanced RP could not be obtained with needle type 
electrodes because the charge density would exceed toxicity limits. They used 
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oxidized iridium as electrode material based on a polyimide substrate. With 250 u.s 
per phase stimulus duration threshold currents were around 1.5 mA for 400 urn 
electrodes in diameter. For 1 ms per phase stimulus duration the stimulus current 
at threshold was between 0.8 and 0.4mA and for 16ms per pulse phase stimulus 
currents at threshold were around 200 uA. Based on these experiments the rheobase 
was calculated as 125 uA with a chronaxie of 2.3 ms per pulse phase. When charge 
densities were calculated they found that for 100 urn electrodes the charge densities at 
threshold were between 4 and 10mC/cm 2 , for 400 urn between 0.28 and 2.8mC/cm 2 
which was larger than their own safety limit (1 and 0.252 mC/cm 2 , resp.). 

In Humayun's series nine blind RP patients were acutely stimulated with either 
platinum wires (25-125 urn diameter) used as electrodes or disc electrodes each 
400 urn in diameter. Charges et threshold were reported between 0.2 and 2.4 p.C 
which gives charge densities between 1 and 96mC/cm 2 . The higher values were 
obtained for needle type electrodes [13]. 



13.5 Spatial Resolution and Pattern Perception 

The main prerequisite for the recognition of forms and patterns is a correct retino- 
topical representation of the stimulus within the visual field. Humayun was able to 
show in his series, that stimuli were correctly identified in terms of their location 
within the retina resp. within the visual field. This was also confirmed by animal 
experiments. Spatial resolution can only be tested with electrode arrays when two 
electrodes or electrode clusters are stimulated simultaneously. The patient has to be 
asked if he sees two spots of light or two distinct patterns. In Humayun's series he 
calculated a spatial resolution of 1.75° which could be achieved with epiretinal 
stimulation [11, 12]. These authors calculated based on simulations by Cha et al. 
[2] that placing a 32 x 32 array over a central field of 0.5 x 0.5 mm onto the macula 
surface would result in a 20/26 Snellen visual acuity. A spacing of 90 |im between 
each electrode would then reduce the visual acuity to 20/200. However, such elec- 
trode arrays are currently not available. Such devices are desirable when AMD 
treatment is supposed to be performed with such implants. 

In a series of experiments in four blind subjects Rizzo and coworkers answered 
the question if blind patients are able to identify a stimulus pattern [15]. Only in 
one out of three patients more than 50% of the given patterns could be identified. 
Two-point discrimination was also tested in this series and only in very few experi- 
ments Rizzo's group was able to find patient responses suggesting the perception 
of two separate objects. In these experiments the electrodes were 1,860 |im apart. 

Acute stimulation experiments were not performed with any camera picture 
input. However, electrodes can be activated as if a certain pattern should be seen 
such as a large letter H. Humayun did such experiments in his series and he found 
that patients were able to detect patterned phosphenes. They used a 25 electrode 
array with a pattern "U" and patient reported a "H" type pattern which the authors 
thought to be the result of a blurring effect due to unstable positioning [13]. 
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13.6 Temporal Resolution 

Experiments to systematically determine temporal resolution in acute tests for retinal 
stimulation have not been reported in detail so far for human subjects. Early results 
indicated that the flicker fusion frequency may be similar for electrical stimulation and 
for normal vision [12]. From animal experiments it should be expected that 25 frames 
per second could be transferred with a retinal implant [6]. It can be expected that data 
on temporal resolution will be extracted from the clinical trials on retinal prostheses, 
which are now being performed. This information is important, because useful vision 
elicited with prosthetic devices does not only depend on spatial characteristics but also 
on the time necessary to transmit a pattern from the implant to the primary visual 
cortex and to identify it. The type of visual sensations that can be obtained. 

If patients are asked after the procedure what they have seen, the descriptions vary 
considerably between individuals. However, a few aspects are common. Usually the 
patients did not experience any unpleasant sensations. With suprathreshold stimulation 
patients reported dots, arcs, circles and lines of different intensity, color, and orienta- 
tion. Stimulation by one electrode may not necessarily result in the perception of one 
single percept or phosphene but also in multiple phosphenes as reported in one volun- 
teer by Rizzo [15]. The size of the objects does also vary with respect to the electrode 
size, the stimulus intensity, and duration. The characteristics of the phosphenes were 
reproducible; i.e. the stimulus pattern X elicits the same phosphene when it was 
repeated at the same retinal area. Weiland and colleagues tested two subjects before 
removal of the eye due to cancer. In these patients they destroyed part of the outer 
retina with argon green and krypton red laser photocoagulation. They stimulated nor- 
mal retina and the laser treated areas with a 125 urn platinum wire electrode. The 
percept after stimulation of the normal retina was a dark oval shaped phosphene 
whereas stimulation in the krypton red treated area revealed a small white spot. 
Stimulation of the argon green treated area resulted in a line type of percept. Stimulus 
threshold at the normal retinal area in their experiments were 0.8 and 4.8 mC/cm 2 
respectively. The percepts being described by blind RP patients were similar to those 
obtained over the krypton red treated normal retinas. From a histological work-up of 
the stimulated retina Weiland concluded that the target for electrical stimulation with 
an epiretinal electrode is not the ganglion cell layer but the inner nuclear layer [19]. 

The shape of the percepts varied significantly with the stimulus pattern or with 
the orientation of the activated electrodes of the array. In Rizzo's series circles were 
seen when columnar electrodes along the axon orientation were activated but also 
curved lines [15]. 



13.7 Subretinal Versus Epiretinal Stimulation 

Acute retinal stimulation experiments in blind humans were only reported for 
epiretinal stimulation, not for subretinal stimulation. Acute tests on subretinal 
stimulation so far have only been reported for rabbits and pigs but not for blind 
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humans [7, 16]. Therefore major assumptions as confirmed for epiretinal stimulation 
were so far not confirmed for subretinal stimulation scenarios. 



13.8 Less Invasive Stimulation Procedures 

Electrical retinal stimulation can also be performed placing electrodes outside the 
eye. It has been shown that even by using corneal electrodes or electrodes placed 
onto the scleral surface it is possible to elicit visual phosphenes. Gekeler and col- 
leagues investigated phosphenes upon electrical stimulation via DTL electrodes 
placed in the fornix. They found that in some RP patients such phosphenes could 
not be elicited. The rheobase for RP patients was 0.69 + 0.10 mA which is 18 times 
higher than for normal individuals [8]. Similar experiments were performed by 
Delbeke and colleagues. They used corneal surface electrodes with large periorbital 
reference electrodes. Their estimation for rheobase varied between 2.14 and 8.16 mA 
and for chronaxie they found values between 0.45 and 0.87 ms whereas for healthy 
volunteers rheobase was 0.28 mA and the mean chronaxie was 3.07 ms [4]. In our 
own experience in patients with advanced stages of retinitis pigmentosa the currents 
necessary to obtain visual percepts with such approaches are very high and close to 
current with which other subjective sensations such as pain or muscle tics were 
evoked. Such approaches are proposed to identify patients prior to surgery who may 
benefit from a chronic implant. Only those patients would be selected in which 
phosphenes can be obtained with such non-invasive techniques. However, we 
learned that in RP patients even when they do not respond to external stimulation or 
the stimulation has to be stopped because of unpleasant somatosensory sensations 
the same patient may have visual sensations with a chronic implant. 1 Therefore, we 
feel that such a test is not useful to identify good candidates for retinal implants. 



13.9 Conclusions and Outlook 

The available data from acute trials in electrical retinal stimulation showed that at 
least with epiretinal stimulation visual responses can be obtained and that the energies 
necessary to achieve such responses are dependent on the material of the electrodes 
and on their sizes and shapes. A close contact between the electrode and the retina is 
desirable to elicit visual responses with low stimulus intensities. A deeper knowledge 
of the mechanisms of retinal stimulation as well as of the phosphenes which are 
elicited and their role in providing vision for a blind patient is necessary. Such 
information can only be obtained by chronic stimulation experiments in which 
enough time is available to study more stimulus parameters, more electrode positions 
over a longer time period and to allow the patients to learn how to interpret the percepts 



'Walter (2005) unpublished observation. 
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induced with a visual prosthesis and to use the potential of visual system plasticity. Such 
data will be provided in the future with semichronic or chronic implants. 

The three crucial questions asked at the beginning of the chapter can now been 
answered: (a) Yes, visual responses can be obtained through electrical stimulation 
of the retina, (b) The stimulus intensity necessary to elicit visual responses 
depends on several factors. Visual responses can be obtained within safe stimulation 
levels in certain scenarios, (c) The two-point discrimination and the perception of 
patterns using such an approach is under debate. In many acute trials there was not 
enough time for retinal stimulation to collect enough data to really answer this 
question. However, the chronic trials, that have already been initiated will answer 
the question in the near future. 
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Chapter 14 

The Perceptual Effects of Chronic Retinal 

Stimulation 

Alan Horsager and lone Fine 



Abstract Can functional vision be restored in blind human subjects using a 
microelectronic retinal prosthesis? The initial indications suggest that, yes, it is 
possible. However, the visual experience of these subjects is nothing like a digital 
scoreboard-like movie, with each electrode acting as an independent pixel. The 
work described here in this chapter suggests that there are interactions between 
pulses and across electrodes, at the electrical, retinal, or even cortical level that 
influence the quality of the percept. In particular, this work addresses the question, 
"how does the percept change as a function of pulse timing on single and multiple 
electrodes"? The motivation for the work described here is that these interactions 
must be understood and predictable if we are to develop a functional tool for blind 
human patients. In this chapter, we review work evaluating perceptual effects using 
chronic electric stimulation in three different implantable systems. 
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MPDA Microphotodiode array 

NLP No light perception 

OCT Optical coherence tomography 

rdl Retinal degeneration 1 

RP Retinitis pigmentosa 

RPE65 Retinal pigment epithelium-specific 65 kDa protein 

SSMP Second Sight Medical Products, Inc. 

VI Primary visual cortex 

VPU Visual processing unit 



14.1 Introduction 

Visual impairment is one of the most common disabilities: at the most recent esti- 
mate, 110 million people worldwide have low vision and 40 million are blind [69]. 
Photoreceptor diseases such as retinitis pigmentosa (RP) and age-related macular 
degeneration (AMD) are responsible for blindness in approximately 15 million of 
those people [15], a number that continues to increase with the aging population 
[18]. Currently, there are no FDA approved treatments for blindness due to photo- 
receptor disease. 

Although a number of highly promising treatments are being developed, each 
suffers from its own set of difficulties. For example, gene replacement therapy 
efforts have made great progress in treating one form of Leber's Congenital 
Amaurosis (an RPE65 mutation) in humans [1, 2, 4, 7, 8, 52]; however, this form 
of RP is relatively rare, and photoreceptor diseases are genetically heterogeneous, 
with single and multi-gene mutations occurring in over 1 80 different genes respon- 
sible for photoreceptor function [19]. For gene replacement therapy to broadly cure 
photoreceptor disease would require at least as many (and, most likely, many more) 
treatments as there are mutations. Another genetic approach uses optical neuro- 
modulators such as channelrhodopsin-2 (ChR2) that can be genetically targeted to 
retinal bipolar [41] or ganglion cells [10, 43] to restore visual responsiveness in a 
mouse model of blindness (rdl). However ChR2 activation requires light stimula- 
tion levels that are 5 orders of magnitude greater than the threshold of cone photo- 
receptors [63], and the induced light responses have a substantially limited dynamic 
range (2 log units) [72]. An ideal therapy would be able to treat blindness indepen- 
dently of the genetic mutation, in the absence of photoreceptors, and with reason- 
able response sensitivity and range. 

Therapies employing direct electrical stimulation of the retina have the potential 
to fulfill those two particular constraints. However, electrical stimulation suffers 
from its own set of limitations. There are a number of engineering concerns such 
as charge density safety limits which limit the miniaturization of implanted elec- 
trodes, difficulties in placing the electrode array close to the target retinal cells, and 
limitations is in the available power supply that make prosthesis design extremely 
challenging. Electric current fields from relatively large electrodes indiscriminately 
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Fig. 14.1 Patient percepts. Example percepts generated by retinal electrical pulse train stimulation 
in two blind subjects, S05 and S06, respectively, using the Second Sight Medical Products, Inc. A16 
epiretinal prosthesis. Percepts (top) were hand drawn by experimenter based upon patient report. 
The electrodes that were stimulated for each condition are shown with solid dots. Stimulation pat- 
terns were 50 Hz pulse trains on each of the electrodes 



drive local retinal circuits in an unnatural way, leading to complex retinal responses. 
Although electrically-driven retinal activation produces phosphenes in blind human 
subjects, these percepts are complex and cannot be simply thought of as a one to 
one, electrode to pixel, scoreboard-like experience with punctate individual phos- 
phenes (Fig. 14.1). 

There is a substantial literature evaluating the use of electrical stimulation to 
generate visual percepts in both sighted and blind human subjects [11, 22, 24, 35, 
37, 42, 44, 45, 50, 53, 54, 61, 62, 75, 77]. However, partly because many of these 
studies were carried out acutely, there has not yet been a been a thorough quantita- 
tive and systematic analysis of how these electric pulses interact within the network 
of retinal neurons in time and space to form the visual image the subject sees. With 
the goal of creating a visual prosthesis that is capable of restoring functional vision 
in blind human patients, much needs to be learned about how the timing of pulses 
(within single electrodes and across multiple electrodes) interact at the electrical, 
retinal, and cortical level to form a percept. 

There is relatively little published data quantitatively examining chronic retinal 
stimulation in human subjects. To date, only three commercial groups have been 
able to collect chronic data: Retinal Implant AG, Intelligent Medical Implants 
GmbH, and Second Sight Medical Products, Inc. To summarize their collective 
findings: (1) an electrode array can be safely and chronically implanted in human 
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patients (more than 5 years as of this writing), (2) stimulation via these electrodes 
consistently produces visual percepts, (3) provided the array is stable on the retinal 
surface, the current-brightness relationship is stable, repeatable, and monotonic, 
(4) the brightness of the percept can be controlled through both amplitude and 
frequency modulation, and (5) signal integration during single or multi-electrode 
stimulation can be approximated using very simple models. In the last part of the 
chapter we discuss the use of these implants during "real world" and mobility tasks. 
While findings are promising, it is still not demonstrated that the devices that are currently 
available can provide useful function vision outside of the laboratory setting. 



14.2 Overview of Chronic Retinal Implant Technologies 

The earliest documented electrically generated percept in a blind human patient 
was in 1755, when Charles LeRoy, a French chemist and physician, discharged a 
Layden jar and supplied electrical current to a brass coil that wrapped around the 
head of a blind man [42]. In addition to "provoking terrible cries [47]", the young 
patient perceived a flame that rapidly descended before his eyes. This is, more than 
likely, the first documented visual phosphene perceived by a blind subject via elec- 
trical stimulation. 

Despite this somewhat unpromising beginning, restoring functional vision using 
electrical stimulation has been a goal of ophthalmologists and vision scientists 
for more than a century. The inspiration for these studies comes from very early 
(and probably inadvertent) electrical activation of visual cortex during neurosur- 
gery. However, it wasn't until the middle of the twentieth century that scientists and 
clinicians began to investigate, more deliberately and rigorously, the relationship 
between electrical stimulation of neural tissue and visual perception. 

In recent years much of the effort in developing a visual prosthesis for the blind 
has focused on electrical stimulation of the retina. There is a substantial amount of 
neural processing within the LGN [5, 20, 73] and VI [34] that transforms the visual 
signal in ways that are complex, nonlinear, and poorly understood. Targeting stimu- 
lation as early as possible in the visual pathway allows one to maximize the use of 
the innate computational processing of the visual system. 

Even within the retina, a variety of approaches have received attention. Retinal 
stimulation devices have been developed for both subretinal (between outer retinal 
layers and the choroid) and epiretinal (between inner retinal layers and the vitreous 
humor) activation. Here, we provide a brief overview of the basic technology and 
the types of psychophysical and behavioral studies that have been conducted with 
subretinal and epiretinal devices. 



14.2.1 The Retinal Implant AG Microphotodiode Prosthesis 

The Retinal Implant AG device has two subretinally implanted components. The 
first is a wire-bound microphotodiode array (MPDA), consisting of approximately 
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Fig. 14.2 (a) General schematic of Retinal Implant AG device. Note the placement of the device 
in the subretinal space, (b) Close-up of the microphotodiode array (MPDA). Permission for repro- 
duction provided by Retina Implant AG 



1,500 photosensitive cells on a 3 mm 2 surface (Fig. 14.2). Each cell unit contains an 
amplifier and electrode, spaced 70 urn apart. The amplitude of the electrical signal 
across each electrode is proportional to the overall illumination of the specific pho- 
tosensitive cell. The second implanted component consists of a 4 x 4 array of 50 |im 
electrodes (with 280 |im spacing) that can be used for direct stimulation (DS). The 
stimulation presented on the electrodes of the DS array can also be independently 
controlled, and each of the parameters (e.g., pulse width and amplitude) can be inde- 
pendently modulated. The MPDA and DS arrays are positioned on a small polyimide 
foil surface and are powered via a transchoroidal, transdermal line. 

This MPDA device was implanted in the fovea of one eye of seven patients blind 
from RP (all seven patients had the MPDA array and six had, in addition, the DS 
array). Devices were chronically implanted for a total of 4 weeks [78]. With the 
exception of one, patients were explanted at the end of this time. Visual perception 
and performance was evaluated in the following ways using DS: (1) the brightness 
of a biphasic pulse (1-2.5 Volts (V), 3 milliseconds (ms) per phase) was assessed 
using a rating scale from 5 (very bright) to (no perception), (2) subjects were 
asked to discriminate stimulation of rows and columns of electrodes in a vertical 
vs. horizontal discrimination task, (3) motion discrimination for sequential stimula- 
tion of electrodes, (4) and subjective reporting of the apparent size of percepts. 
Three additional patients were implanted at a later date with a similar device 
(for this device the stimulating electrodes were 100 urn). With these three patients, 
the researchers conducted more complex visual perception tasks such as letter rec- 
ognition and orientation discrimination [81]. Data collected on all ten patients using 
this device are described in Sects. 14.4.1 and 14.5.1. 



14.2.2 The Intelligent Retinal Implant System 



The Intelligent Retinal Implant System™ has two external components (a Visual 
Interface and the Pocket Processor), and a subretinally-implanted Retinal Stimulator 
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Fig. 14.3 Schematic of the 
Intelligent Retinal Implant 
System. Illustration kindly 
provided by IMI 




(Fig. 14.3) designed by IMI Intelligent Medical Implants. The Visual Interface 
consists of a pair of glasses with a camera to capture the visual image and other 
components for data communication with the Pocket Processor and Retinal 
Stimulator. Communication with the Retinal Stimulator is conducted via wireless 
transmission. The Pocket Processor supplies power to the entire system and con- 
tains a microcomputer that translates the image data into the stimulation protocol 
for the Retinal Stimulator. The internal flexible Retinal Stimulator consists of a 49 
electrode array and is attached using a silicon ring to a titanium tack which had 
been placed in the sclera. 

Four patients (56-66 years old) with visual acuity ranging from no light percep- 
tion (NLP) to hand movement were implanted. Approximately 20 different testing 
sessions were conducted with the patients over a 12 month period. Each testing 
session consisted of single or multi-electrode pulse train stimulation. In these 
patients, performance on absolute threshold, point discrimination, and pattern rec- 
ognition tasks was measured. See Sects. 14.4.2 and 14.5.2 for details regarding 
psychophysical data collected using this system. 



14.2.3 Second Sight Medical Products, Inc. A16 System 



The Second Sight Medical Products, Inc. (SSMP) A16 epiretinal prosthesis con- 
tains similar intraocular (electrode array) and extraocular (e.g., glasses, Visual 
Processing Unit) components as the Intelligent Retinal Implant System™. The 
intraocular array consists of 16 platinum electrodes in a 4x4 arrangement, held in 
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place within a clear silicone rubber platform [37, 45]. The electrode array is 
implanted epiretinally in the macular region and held in place using a retinal tack. 
Electrodes are either 260 or 520 urn in diameter (subtending 0.9° and 1.8° of visual 
angle, respectively). Electrodes are spaced 800 urn apart, center to center. 

Pulse train signals are generated and sent to an external Visual Processing Unit 
(VPU) using custom software run on a PC laptop. Power and signal information are 
sent from this processor through a wire to an external transmitter coil that attaches 
magnetically, and communicates inductively, to a secondary coil that is implanted 
subdermally in the patient's temporal skull. From this secondary coil, power and 
signal information are sent through a subdermally implanted wire that traverses the 
sclera to the array of electrodes (Fig. 14.4). Stimulation can be presented using two 
different protocols: (1) camera mode - real-time video captured by a miniature 
video camera mounted on the subject's glasses is continuously sampled by the VPU 
and a monotonic transform determines the stimulation current amplitude in each 
electrode based on the (normalized) luminance at the corresponding area of the 
scene and (2) direct stimulation mode - the stimulation signal sent to each electrode 
is independently controlled by the VPU or an external computer. 

Six patients have been examined that were chronically implanted with the A16 
retinal prosthesis. See Table 14.1 for details regarding these subjects. Testing ses- 
sions lasted a maximum of 4 h with frequent rest periods. Testing sessions included 
threshold and impedance measurements as well as other measures of visual perfor- 
mance reported elsewhere [77]. When performed, threshold measurements were 
usually carried out at the beginning of a given testing session. The frequency of 
testing sessions was limited by the subjects' availability and the clinical trial protocol. 
In general, testing was carried out 1-2 sessions/week for each subject. The protocol 




Fig. 14.4 (a) Electrode array. The electrode array consisted of 260 or 520 micrometers (|^m) 
electrodes arranged in a checkerboard pattern, with center-to-center separation of 800 \xm. 
The entire array covered -2.9 mm by 2.9 mm of retinal space, subtending -10° of visual angle, 
(b) Prosthesis system schematic. The stimulus sets are programmed using Matlab® on a PC, 
which then communicated the stimulus parameters to an external Visual Processing Unit (not 
shown). Signal and power information was then passed through an external inductive cou- 
pling device (not shown) that attaches magnetically to a subdermal coil implanted in the 
patient's temporal skull. This signal is then sent through a parallel system of wires to the 
epiretinally implanted electrode array. Note that the power and signal information can be 
independently controlled for each electrode. Reprinted from [33], with permission 
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Table 14.1 A16 subjects' age at implantation, eye of implantation, preoperative acuity in the 
implanted and non-implanted eyes, and electrode sizes 

Age Eye VA (implanted) VA (non-implanted) Electrode size (|jm) 

520 
520 
260 

260/520 
260/520 
260/520 

Where there was a difference in pre-operative vision between the two eyes, implantation was 
carried out in the eye with worse vision. One subject (S02) had two operations in the same eye 
since her electrode array detached from the retina after 1 1 months due to the subject falling and 
bumping her head (no retinal detachment occurred). In the second surgery, the electrode array was 
reattached in a nearby macular area no more than 500 p.m distant from the position of the original 
implant. Testing for S01 was limited in duration due to geographical constraints. Testing for S03 
ended due to medical reasons unrelated to the implant. Testing in S04 was ended after microper- 
foration of her conjunctiva which led to cable exposure. Because her cardiac status had deterio- 
rated since the initial implantation she could not undergo anesthesia. This prevented the use of a 
scleral patch graft to repair the microperforation. To minimize the risk of possible infection, the 
multi-wire cable connecting the electrode array to the extraocular stimulator was cut and the 
electrode array was left in place 



specified that optical coherence tomography (OCT) measurements were only 
carried out on the subjects for clinical reasons, and as a result OCT data were only 
collected at irregular intervals. 

The bulk of the data described in this chapter were collected using this epiretinal 
device. As a result more detailed information is given about the patients implanted 
with this device (Sects. 14.3, 14.4.3, and 14.5.3). 



14.3 Thresholds on Individual Electrodes 

One major concern in the field of retinal prostheses is that the current amplitude 
required to elicit percepts may fluctuate unpredictably over time, due to neurophysi- 
ological changes of the retina due to reorganization [40], electrochemical changes on 
the electrode surface, or instability of position of the electrode array on the retinal 
surface [21, 77]. Previous acute studies found that localized retinal electrical stimula- 
tion of blind subjects resulted in discrete visual percepts; however, the amount of 
electrical current required to elicit visual responses was relatively large compared to 
animal in vitro retinal studies examining responses to electrical stimulation [36, 61]. 
The most exhaustive examination of thresholds and brightness reported to date 
has been within the six subjects implanted with the A16 epiretinal prosthesis 
(Second Sight Medical Products, Inc.). Over the course of several years, we measured 
the distance of the electrodes from the retinal surface (using Optical Coherence 
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Tomography, OCT), retinal thickness, electrode impedance, and perceptual thresholds 
for both single pulse and pulse train stimuli [21, 33]. These data allowed us to examine 
the relationship of perceptual threshold to electrode size, electrode impedance, 
distance of the electrodes from the retinal surface, and retinal thickness. 



14.3.1 Single Pulse Thresholds Using the SSMP System 

Thresholds were measured on single electrodes using a single interval, yes-no 
procedure. On each trial, subjects were asked to judge whether or not a stimulus 
was present. This reporting procedure meant that subjects were likely to report 
stimulation for either a light or dark spot; subjects were explicitly instructed to 
include either type of percept in making their decision. Half of the trials were 
stimulus-absent catch trials. Current amplitude was varied using a three-up-one- 
down staircase procedure to find the threshold current amplitude needed for the 
subjects to see the stimulus on 50% of stimulus-present trials, corrected for the false 
alarm rate. During each staircase, only amplitude varied. All other parameters 
(frequency, pulse width, pulse train duration, and the number of pulses) were held 
constant. Thresholds were measured for each of the 16 electrodes using a single 
"standard pulse" consisting of a 0.975 ms cathodic pulse followed by a 0.975 ms 
anodic pulse. 

Thresholds measurements for each subject are shown in Fig. 14.5. Differences in 
threshold did not change systematically as a function of patient age or pre-operative 
vision [76]. However, thresholds did appear to decrease as a function of successive 
surgeries. Generally, thresholds decreased across subject implantations. Indeed, 
subject SOI had the highest threshold overall. This improvement across surgeries 
was perhaps due to the overall improvement of the surgical procedure, leading to 
the electrode array lying successively closer to the retinal surface. For subjects S05 
and S06, most of the measured single pulse thresholds were well below 100 uA and 
charge density limits of 0.35 mC/cm 2 . It should also be noted that these thresholds 
are for a single pulse, whereas functional electrical stimulation is likely to be medi- 
ated by pulse trains, which generally require lower stimulation thresholds (see 
Sect. 14.3.2). 

Mean thresholds for subjects S04-S06 across the 260 and 520 urn electrodes 
were compared to determine if electrode size had any effect on the measured 
values. Interestingly, there was no noticeable difference in threshold between 260 
and 520 urn electrodes, either within or across subjects (two-factor, subject x elec- 
trode size, ANOVA, p > 0.05 F = 0.367), see Fig. 14.6. This was in direct contrast to 
a recent literature review by Sekirnjak et al. who found, across a wide range of 
in vitro and in vivo studies, that log thresholds increase linearly with log electrode 
area [64]. However, only two electrode sizes are evaluated here, and it is possible that 
a wider range of sizes would make threshold differences, as a function of electrode 
size, more apparent. 
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Fig. 14.5 Mean thresholds across the entire time period of implantation (uiA) for all 16 electrodes 
for each subject. The upper panels show the labeling scheme used to identify electrodes, as viewed 
through the pupil. For each subject, electrodes are ordered from most to least sensitive along the 
x-axis. White and black bars represent electrode diameters of 260 and 520 tun respectively. 
Threshold current is shown along the y-axis. Note the dramatic change of scale along the y-axis 
across subjects. Error bars are +/- one standard error of the mean. Reprinted from [21], with 
permission 
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Fig. 14.6 In both (a) and (b), the x-axis represents electrode diameter and the y-axis represents 
the current needed to reach perceptual threshold, (a) is the data are taken from subjects S04-S06. 
The large symbols connected by lines represent the mean threshold across each of the eight elec- 
trodes of a given diameter for each subject. Error bars are +/- one standard error of the mean. In 
many cases error bars are smaller than the symbols. Individual electrodes are shown with small 
symbols, (b) compares our measured thresholds in S04-06 (large open shapes) and those reported 
in the literature [64]. Reprinted from [21], with permission 
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Fig. 14.7 OCT imaging of the array, (a) Fundus photograph of an intraocular electrode array 
viewed through a dilated pupil, imaged hy the OCT machine (OCT, STRATUSOCT™; Carl Zeiss 
Meditec AG) just previous to the cross-sectional OCT image shown in (b). (b) Cross-sectional 
OCT image. Location of electrodes that were imaged are denoted by the letters A, B, C, and D. 
Reprinted from [21], with permission 



The distance from the top of each electrode to the internal limiting membrane 
of the retina varied both within and across subjects, as measured using optical 
coherence tomography (OCT) (Fig. 14.7). Electrode thickness varied between 80 
and 120 urn depending on the exact cross-section, so 100 |im was subtracted from 
the measurement of the distance of the electrode to the internal limiting membrane. 
The thickness of the retina was defined as the distance from the inner surface of the 
retinal pigment epithelium to the internal limiting membrane. 

Impedance was measured using Second Sight Medical Products (Inc.) proprietary 
software. Impedance measurements were taken at the beginning and end of each 
stimulating session. 

Data suggests that distance from the retinal surface is a critical factor in deter- 
mining both threshold and impedance. For a given electrode size, electrodes that are 
close to the retinal surface have lower thresholds and higher impedances. As shown 
in Fig. 14.8b, we see a positive correlation between threshold current and electrode 
distance from the retina. These psychophysical data are consistent with retinal 
electrophysiology data, suggesting that the distance of the electrodes from the 
retina is a significant concern [31, 38]. Stimulus current requirements are likely to 
be minimized when the array is in close position to the retina, minimizing power 
consumption by the stimulator and allowing for smaller electrodes to generate 
phosphenes within safe charge density limits. 

On the whole, subject impedances tended to decrease postoperatively over time. 
However, impedances are also negatively correlated with the distance of the elec- 
trode from the retinal surface, as shown in Fig. 14.8c. These data are consistent with 
the notion that electrodes that are close to the surface of the retina have higher 
impedances (due to the adjacent retinal tissue) than electrodes that have lifted from 
the retina (where fluid with higher conductivity intervenes between the electrode 
and the retinal surface) [32, 65]. Indeed, consistent with this hypothesis, threshold 
is negatively correlated with the impedance (Fig. 14.8a). 
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Fig. 14.8 Correlations between threshold, impedance, electrode distance and retinal thickness. 
Each subject is shown with a different symbol shape. Straight lines represent the best fitting linear 
regression on log-log axes, (a) Impedance versus threshold, (b) Electrode distance from the retinal 
surface vs. threshold. The curved solid lines show predicted thresholds for 260 (lower, thin solid 
line) and 560 u.m (upper, thick solid line) electrodes based on the model of Palanker et al. [51]. (c) 
Electrode distance from the retinal surface vs. impedance, (d) Distance versus retinal thickness, 
(e) Retinal thickness versus threshold, (f) Retinal thickness versus impedance. Reprinted from 
[21], with permission 



14.3.2 Pulse Train Integration and Temporal Sensitivity 



It is important to understand, at the single electrode level, how the electrical signal 
is integrated over time to modulate visual sensitivity or suprathreshold brightness. 
The retina delivers information about the visual scene to higher visual centers 
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through its time-varying spike signal [3, 23], so it would be potentially beneficial 
to encode the light signal using pulse sequences that trigger "naturalistic" 
patterns of activity in retinal cells. Indeed, it has been shown that using short 
electrical pulses results in phase-locked spikes in ganglion cells up to 250 Hz 
[25, 64]. 

Until recently, the general assumption has been that the rate of ganglion cell firing 
is simply monotonically related to the "intensity" of the stimulus. However, this 
idea of "simple rate coding" has recently come into question as it has been shown 
that the visual system is sensitive to spike timing on a much finer scale (<10ms) 
[60] and spike train variability cannot simply be described by a Poisson distribution 
[9, 70]. Indeed, the data in this section show that the human visual system is not 
only highly sensitive to changes in overall pulse train frequency, but to the distribu- 
tion of the pulses within a given window of time. Although we can only speculate 
as to the underlying physiological mechanism that is involved in integrating these 
pulse signals, it clearly shows that being able to control the precise timing of spikes 
may prove to be as important as controlling their absolute rate. 

The most comprehensive study of temporal integration comes from patients 
implanted with the Second Sight Medical Products, Inc. epiretinal prosthesis [33]. 
We describe here a recently published model suggesting that visual sensitivity for 
electrical stimulation can be described by a relatively simple linear-nonlinear 
model that predicts the relationship between electrical stimulation and brightness 
for any temporally-varying stimulation pattern. This model can not only be used to 
determine the "optimal" pattern of stimulation given a variety of engineering con- 
straints (such as stimulating at safe levels of charge density and minimizing overall 
charge, for example), but in addition, its biological plausibility provides insight into 
the neural pathways that underlie the perceptual effects of electrical stimulation. 
Data described in this section were collected on two subjects (S05 and S06) using 
the Second Sight Medical Products, Inc. retinal prosthesis. 

Threshold values were collected as described in Sect. 14.3.1. Suprathreshold 
brightness-matching was carried out on single electrodes using a two-interval, 
forced-choice procedure. Each trial contained two intervals with each interval con- 
taining a pulse train of a different frequency. For example, interval 1 might contain 
a 15 Hz pulse train and interval 2 might contain a 45 Hz pulse train. Subjects were 
asked to report which interval contained the brighter stimulus. A one-up, one-down 
staircase method was used to adjust the amplitude of the higher frequency pulse 
train based on the observer's response. Using this method, we were able to obtain 
an isobrightness curve that represented the current amplitude needed to maintain 
the same subjective brightness across a wide range of frequencies. 

Data were modeled using a linear-nonlinear model (Fig. 14.9) similar to models 
of auditory stimulation in cochlear implant users [66], retinal ganglion cell spiking 
behavior during temporal contrast adaptation [6, 16, 59], and human psychophysi- 
cal temporal sensitivity in normal vision [74]. The stimulus was convolved with a 
temporal low-pass filter using a one-stage gamma function: 

r l (t) = f(t)*S(t,l,T 1 ) (14.1) 
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Fig. 14.9 Model schematic. The time varying stimulus is convolved with a linear filter. The result 
of this convolution is passed through a static nonlinearity and convolved with a secondary linear 
filter. We assumed that a stimulus was at visual threshold (or a given brightness level) when 
threshold reached a specific value. Reprinted from [33], with permission 

where f(t) is the electrical stimulation input pattern, t is time (ms), and 8 is the 
impulse response function with time constant x r The gamma function used to 
model this impulse response can be generally described as: 



8{t,n,X x )- 



TjOi-1)!! x 



\"~ 



(14.2) 



where r=time, n = the number of identical, cascading stages, and x is the time con- 
stant of the filter (the one-stage gamma function in (14.1) is simply an exponential 
function). 

We assumed that the system became less sensitive as a function of accumulated 
charge. This was computationally implemented by calculating the amount of accumu- 
lated cathodic charge at each point of time in the stimulus, c(t), and convolving this 
accumulation with a second one-stage gamma function having a time constant x 2 . The 
output of this convolution was scaled by a factor e and then subtracted from r. (14. 1), 



r 2 (0 = r,(0-e(c(0*5(M,T 2 )). 



(14.3) 



r 2 was then passed through a power nonlinearity, 

r 3 (t) = (r 1 (t)f 



(14.4) 



and convolved with a low-pass filter described as a three-stage gamma function 
with time constant x„ 



r 4 (0 = i- 3 *5(r,3,T 3 ). 



(14.5) 



We assumed that the response reached threshold (or the point of equibrightness 
during suprathreshold experiments) when 



max t(r 4 )>— 6 



(14.6) 



where 9 is a fixed constant. 
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Patients typically reported that phosphenes appeared white or yellow in color, 
and round or oval in shape. At suprathreshold, percepts were reported as brighter 
and the shape occasionally became more complex than a simple round or oval 
shape. The shapes were reported as being approximately 0.5-2 in. in diameter at 
arm's length, corresponding to roughly 1-3° of visual angle. Occasionally, a dark 
spot rather than a white or yellow percept was reported. In this case, the patient 
would use the relative contrast of the spot for detection (threshold) or "brightness 
comparison" (suprathreshold). 

After optimizing the model using a subset of the full set of data, the best-fitting 
parameters values were averaged for z v x 2 , x 3 , £, and P across the electrodes used 
for optimization. These mean values were then used to predict threshold and 
suprathreshold data for novel electrodes. 

Figure 14.10 shows subject thresholds (gray squares) and model predictions 
(solid line) for a single biphasic pulse presented on a novel electrode for both sub- 
jects. Figure 14.11 shows threshold data and predictions for pulse trains containing 
either 2 (B) or 15 (C) pulses, whose frequency was varied between 3 and 3,333 Hz. 
In summary, the model and parameter values generalized to successfully predict data 
on novel electrodes. The data shown in these two figures are from a subset of over 
ten experiments conducted over five electrodes from each of the two subjects. 

The ability of the model to predict suprathreshold responses to novel pulse train 
waveforms not used to optimize model parameters was then examined (Fig. 14.12). 
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Fig. 14.10 Single pulse threshold. These data are from electrodes C3 and Al, from patient S05 
and S06, respectively. Stimuli (a) were single, biphasic, charge-balanced square pulses, whose 
pulse width (dashed arrow) varied in duration from 0.075 to 4 ms. For each pulse width, the ampli- 
tude was varied (solid arrow) to determine perceptual threshold. In the data plots (b), the x-axis 
represents pulse width (plotted logarithmically) and the y-axis represents the current amplitude 
(uA) needed to reach threshold. The solid black line represents the prediction of the model. 
Reprinted from [33], with permission 
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Fig. 14.11 Variable duration pulse train threshold. These data are from electrodes C3 and Al , from 
patient S05 and S06, respectively. Stimuli (a) were pulse trains whose frequency was varied between 
3 and 3,333 Hz. Pulse trains contained either 2 (b) or (c) 15 pulses. The amplitude of all pulses within 
the train was varied simultaneously to determine threshold (see Methods for a full description of the 
threshold detection task). The x-axis represents pulse train frequency (Hz) (plotted logarithmically) 
and the y-axis represents the current amplitude (\iA), per pulse, needed to reach threshold. The 
black line represents the prediction of the model. Reprinted from [33], with permission 



Again, the same fixed values for tj, x 2 , x,, s, and P based on the electrodes and 
stimulus patterns used for optimization were used, and the only parameter allowed 
to vary across each experiment was the threshold parameter 8. The novel wave- 
forms consisted of repeated bursts of three pulses with a variable inter-burst delay. 
The model and parameter values generalized to successfully predict these data from 
a novel stimulation pattern on a novel electrode. 

This model, like those describing the perception of light stimuli, presumably 
approximates the responses of neuronal populations. In the case of our threshold 
experiments, it is possible that firing within a relatively small number of retinal 
cells mediated detection. It has been previously shown that subjects with normal 
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Fig. 14.12 Bursting pulse triplets, suprathreshold. These data are from electrodes Al and A2, 
from patient S05 and S06, respectively. All pulse train stimuli (a) were either 15 (b), 30 (c), or 60 
(d) pulse trains that were 500 ms in duration, consisting of bursts, or triplets, of groups of three 
pulses. Each burst consisted of 0.45 ms biphasic pulses with no inter-phase delay. The x-axis 
represents the inter-pulse delay between the set of three bursting pulses (plotted logarithmically), 
and the y-axis is current amplitude (|^A), per pulse, needed to reach equibrightness. All stimuli 
were brightness matched to the maximally separated, or evenly distributed, pulse trains (32.4 ms 
delay for (b), for example). The black line represents the prediction of our model. Reprinted from [33], 
with permission 
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vision can reliably detect a single photon of light [30], suggesting that a very small 
increase over the baseline firing rate of ganglion cells is probably sufficient to mediate 
behavioral detection. Thus, thresholds in our subjects may have been mediated by 
a relatively small number of spikes: these spikes might, of course occur either in a 
single cell or across several cells. At suprathreshold our model presumably approxi- 
mates the population response of a larger number of cells each producing one or 
multiple spikes. 

Tj (14.1). The parameter X, represents the time course of the first stage of current 
integration. Estimates of x, in our model vary between 0.24 and 0.65 ms, with a 
mean of 0.42 ms, a value very similar to electrophysiology estimates of the integra- 
tion of current by ganglion cells [25, 39, 64]. In contrast, long-latency spiking in 
ganglion cells, occurring >8-60 ms after the beginning of electrical stimulation 
[25, 27, 39], originate from bipolar cells since a cocktail of synaptic blockers com- 
pletely suppresses this late-phase spiking in ganglion cells. The time constant associ- 
ated with the inhibitory input from amacrine cells is on the order of 100-200 ms 
[25]. The similarity of x, to time constants of current integration by ganglion cells 
suggests that direct stimulation of ganglion cells (rather than indirect stimulation via 
pre-synaptic input) may be primarily responsible for integration of stimulation current 
within the retina, particularly with pulse widths longer than 1 ms [27]. 

sand T 2 (14.3). The parameters £ and T 2 represent desensitization as a consequence 
of accumulated charge, where £ represents the strength of desensitization and T 2 
represents the time constant over which charge was integrated. There are two pos- 
sible sources for this change in sensitivity. One possibility is that injected charge 
directly results in a hyperpolarization of the membrane resting potentials within 
individual ganglion cells. Shifts in resting potentials, analogous to slow contrast 
adaptation effects, can be produced in ganglion cells by injection of hyperpolarizing 
current [6]. However, it is as likely that inhibition from presynaptic cells was 
involved in the desensitization we observed. Inhibitory presynaptic influences on 
spiking in response to electrical stimulation have been described by Fried et al. 
[25], particularly for longer pulses. It seems likely that the desensitization stage of 
our model simply approximates a series of complex adaptive processes, with time 
courses varying between milliseconds to tens of seconds [6, 16, 59]. 

/3 (14.4). (3 describes a power input-output nonlinearity. Power nonlinearities are 
frequently used in linear-nonlinear models describing spiking behavior in ganglion 
cells [6, 16]. A similar nonlinearity has been used in modeling human behavioral 
data of light stimuli [74]. One possibility is that as the intensity of stimulation 
increases, neurons with shallower input-output nonlinearities are recruited. 
Alternatively, this change in the power function may be driven by changes in the 
input-output nonlinearity within individual cells. It has been found in models of 
retinal spiking that the slope of the nonlinearity changes as a function of increased 
contrast [6, 59]. 

Tj (14.5). x 3 determines the integration period of the final low pass filter. Thresholds 
decrease as a function of frequency for a fixed number of pulses, with an asymptote 
at around 100-200 Hz, with the effect being most noticeable for the pulse train 



14 The Perceptual Effects of Chronic Retinal Stimulation 289 

containing 15 pulses, x, may represent the slow temporal integration that occurs in 
cortex. Similar integration times have been found in simple cell recordings in cat 
striate cortex [55]. 

A successful retinal prosthesis will need to produce percepts consisting of regions 
of constant brightness across a range of brightness levels, while satisfying a complex 
set of engineering constraints: charge densities must remain relatively low, it is tech- 
nically difficult to produce very high current amplitudes, and absolute charge must 
be minimized to maximize battery life. Models of the perceptual effects of electrical 
stimulation, such as that described here, will be critical in allowing electrical stimu- 
lation protocols to be selected that best satisfy these many constraints. 



14.4 Suprathreshold Brightness 

A visual prosthesis should produce regions of constant brightness across a range of 
brightness levels, and ideally these suprathreshold brightness levels should be con- 
sistent with the apparent brightness of objects as they appear to those with normal 
vision. To date, all three groups have examined how apparent brightness changes as 
a function of stimulation intensity. 



14.4.1 Brightness Using the Retinal Implant AG System 

Brightness as a function of stimulus intensity has been measured in patients 
implanted with the Retinal Implant AG device [78, 80]. These tests have tended to 
use a slightly more clinical methodology than the psychophysical measures 
reported for patients implanted with the Second Sight LLC implant. Among other 
tests (described below), patients were asked to rate the perception of brightness 
elicited by applying biphasic voltage impulses from 1 to 2.5 V presented on four 
electrodes in a square configuration (3 ms pulse duration, presented in a random 
order) using a scale from 5 (very strong) to (none). When there were six steps 
between 1 and 2.5 V (corresponding to a charge increase of approximately 0.23 mC/cm 2 
between each stimulus assuming a linear scale) the apparent brightness of the elic- 
ited spots varied from scale to 5 in a linear manner. 

This group has also carried out brightness matching experiments using pairs of 
pulses that differed by as much as 0.8 V (10 s interval between each pulse). A dif- 
ference in brightness between two consecutive pulses was discerned if a difference 
in charge of at least 161 |iC/cm 2 was applied. If equal charges were applied within 
both stimulation intervals, the second flash always was perceived as slightly dimmer 
irrespective of the stimulation level. 

Subjective brightness-size interactions were observed at medium stimulation 
levels and at certain frequencies. The subjective size of the phosphene elicted by 
four electrodes increased from 1 to 5 mm (at arm's length) if the voltage was 
increased from 1.5 to 2.5 V [79]. 
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14.4.2 Brightness Using the Intelligent Medical Implant System 

Thresholds for each patient were obtained from a Weibull fit and resulted in an average 
threshold of 25.3 ±7 nC, which is considerably lower than the result obtained in a 
previous study involving acute stimulation with the same patients [57]. In one sub- 
ject, thresholds were measured over a period of 4 months. In this case, thresholds 
were determined by stimulating at charge levels between and 122 nC [58], using a 
two-alternative forced-choice procedure. In total, 23 runs were conducted over the 
course of the 4 months. Although the threshold varied from 8.0 to 35.9 nC between 
subjects, the data suggest that the thresholds were stable over the entire 4 month 
period. Visual percepts depend on amplitude levels and electrode location [57]. 



14.4.3 Brightness Using the SSMP A16 System 

In these measurements, two subjects implanted with the A16 epiretinal prosthesis 
were asked to rate the brightness of a test pulse in comparison to the brightness of 
a standard of fixed current amplitude. Stimulation for test and reference pulses 
always consisted of a single biphasic, cathodic-first, charge-balanced square wave 
pulse, with a pulse duration of 0.975, and a 0.975 ms inter-pulse interval. The refer- 
ence pulse was fixed at a current amplitude chosen to be roughly 2.5 times the 
threshold amplitude for a single pulse on that electrode. 

We used a classic brightness matching procedure based on that of Stevens [68]. 
Before beginning each testing session, subjects were repeatedly stimulated with the 
reference pulse and were told, "This reference pulse has brightness of 10 and we will 
present it to you before we begin each trial. Your task is to compare the brightness of 
the test pulse in each trial to the brightness of this reference pulse. If the test pulse 
seems to be twice as bright as the reference pulse then give it a rating of 20. If the test 
pulse seems to be half as bright as the reference pulse, then give it a rating of 5." 

Once the subject reported feeling confident of having a clear idea of the bright- 
ness of the reference pulse, we began the experiment. All subject ratings were 
provided verbally. On each trial, subjects were first presented with the reference 
pulse and were reminded that this pulse should be considered as having a brightness 
of 10. This reference pulse was quickly followed by the test pulse. Subjects were 
then asked to verbally rate the apparent brightness of the test pulse, as compared to 
the reference pulse (Fig. 14.13). 

The test pulse was always presented on the same electrode as the reference 
pulse, and had a current amplitude that varied pseudo-randomly from trial to trial 
using the method of constant stimuli. Subjects were not told which test pulse cur- 
rent value had been presented on each trial, and no feedback was provided. Each 
test current amplitude was presented four times, and the mean and the standard 
error of brightness ratings for each stimulation amplitude was calculated across 
these four repetitions. 
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Fig. 14.13 Brightness matching data for both subjects. For each subject, equibrightness measure- 
ments were conducted at with pulse amplitude on the reference electrode fixed at five different 
amplitude levels. The x-axis represents the amplitude of the pulse on the reference electrode. The 
y-axis represents the PSE on each of six test electrodes and the reference electrode brightness 
matched to itself. 520 urn electrodes are represented by large symbols, 260 |^m electrodes by small 
symbols. The dashed line represents equal amplitude on test and reference electrode. The different 
labels represent measurements for a specific electrode (e.g., CI). Reprinted from [28], with 
permission 



Brightness matching judgments were also carried out, subjects made brightness 
judgments (which interval contains the brighter stimulus) between a pulse train 
presented on a reference electrode and a pulse train presented on a test electrode 
using a two-interval forced choice procedure. Intensity of the test electrode was 
adjusted through a staircase procedure and data were fit with a cumulative normal 
distribution to find the point of subjective equality. Subjects could reliable differen- 
tiate between pulse pairs separated by less than 20 uA in the discrimination 
paradigm. 

Both brightness rating and brightness discrimination judgments could be well fit 
by a classic Stevens function, B = aC b , where B is the brightness rating made by the 
subject, C is stimulus current amplitude, and a and b are free parameters. Data 
could still be fit when b was fixed to be the median of the best-fitting values of b 
across all four electrodes for that subject, suggesting that it may be possible to 
normalize brightness across an entire array of electrodes by measuring a single 
parameter for each electrode. 



14.5 Spatial Vision 



The data described above show that it is possible to control the perceptual bright- 
ness of a stimulus presented on a single electrode through either the timing or ampli- 
tude of stimulation. The data described below examine how multiple electrodes 
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interact to create a patterned image, and the first studies examining whether these 
implants might offer the potential to provide functional vision. 

When evaluating spatial vision or functional vision tasks it is important to rec- 
ognize that subjects may develop "strategies" to perform these tasks, especially 
when given previous training with feedback within tasks that have a constrained 
set of response alternatives. This is a particular concern for two or four alternative 
forced choice tasks with training and/or feedback. These are of course perfectly 
valid psychophysical techniques, and these tests are based on standard clinical 
tests of visual acuity, but it needs to be remembered that in these cases the subject 
is performing a constrained discrimination task, not an identification task. For 
example, these tasks may not be particularly revealing about whether subjects are 
doing these tasks on the basis of percepts that would be meaningful outside the 
laboratory environment. To take an extreme case, if stimuli are "jumbled" in 
space, then each of two alternatives might produce a pseudo-random percept, but 
these percepts would be perceptually distinct. In this case the subjects could per- 
form the task perfectly with training but the implant would be useless for func- 
tional vision. 



14.5.1 Spatial Vision with the Retinal Implant AG System 

As described above, a set of computerized, standardized tests for patients with 
visual prostheses was developed to quantify the functional abilities of patients 
implanted with the Retinal Implant AG device [78, 80]. 

Electrical stimulation of rows, columns and blocks of four electrodes allowed 
some patients to clearly distinguish horizontal from vertical lines under four-alternative 
forced choice conditions. Under optimal conditions, dot alignment and direction of 
dot movement could also be differentiated when three neighboring electrodes were 
switched on simultaneously or sequentially at 1 s intervals. 

This study also reports evidence examining letter-reading and stripe pattern 
recognition using the Retinal Implant AG system. Using the direct stimulation (DS) 
4x4 array, electrode stimulation configurations were used to represent letters. 
These images were perceived as 5 cm in diameter when presented at a 60 cm dis- 
tance [80]. Patient 1 correctly determined the orientation of a letter "U" (20/24 
times) when using a four alternative forced-choice task (4 AltFC). Patient 2 cor- 
rectly discriminated between the letters C, O, I, L, and Z using this same 4 AltFC 
paradigm. Additionally, when using the light-sensitive chip, this same subject was 
able to differentiate the letters L, I, T, and Z when presented on a screen 62 cm 
away. Both Patient 1 and Patient 3 could determine the direction of lines or stripe 
patterns using the light sensitive chip (11/14 and 11/12, respectively). To date it is 
not clear why some subjects could perform some tasks, but not others, and to what 
extent performance on these tasks was mediated by practice with feedback with 
individual stimuli. 
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14.5.2 Spatial Vision with the Intelligent Medical 
Implant System 

The Intelligent Retinal Implant (IRI) was implanted in four patients with bare light 
perception (BLP) or less. These patients were then tested on at least 20 separate 
occasions over a period of approximately 12 months. Across these session both 
thresholds and pattern recognition was evaluated. 

During stimulation sessions the patients were able to distinguish between differ- 
ent points in space when spatially-segregated electrodes were activated. This point- 
to-point discrimination task was successful both horizontally and vertically. When 
presenting multi-electrode stimulation, patients were able to recognize simple 
patterns such as horizontal bars [56] in a forced choice procedure. Simple patterns 
(vertical/horizontal bar, a cross) administered via activation of appropriate elec- 
trodes were also distinguishable by patients in forced choice procedures. 



14.5.3 Spatial Vision with the SSMP A16 System 

Assuming that each electrode on a 2-dimensional electrode array can produce indi- 
vidual, punctate phosphenes, visual resolution is simply limited by the pitch or 
spacing of the electrodes. Using the Second Sight A16 system, visual acuity perfor- 
mance was evaluated in a single blind human subject (S06) to determine whether 
his spatial visual resolution could approach the level expected from the spacing 
between the electrodes in the A16 electrode array [14]. 

The first experiment tested whether or not an oriented contour could be generated 
using the retinal prosthesis in direct stimulation mode (see Sect. 14.2.3 for a more 
in-depth description of the direct stimulation and camera modes). In each trial, a 
single row of 4 electrodes was used to stimulate a row and then a column (with a 1 
second delay between the two stimuli), and the subject was instructed to draw on a 
board the pattern they perceived. The predicted percept would be a right-angle cross 
of the two lines with a 90° angle of intersection. A head-mounted camera system 
was used to record the movement of the marker on the board at arm's length from 
the subject and the digital data output was analyzed offline. Over 14 trials, the sub- 
ject drew 2 lines with an average angle of 87.4° (1.8° standard error). 

In a second experiment, S06 was asked to report the orientation of a high-contrast, 
square-wave grating presented on a screen. The orientation of these gratings was 
either horizontal, vertical, diagonal right orientation, or diagonal left orientation. 
Thus, the chance performance on this task was 25%. These data were collected in 
camera mode. In each session, high-contrast gratings of different spatial scales 
(2.77-2.00 logMAR; Snellen equivalent 20/11,777-20/2000) were randomly inter- 
leaved. The probability of detection was calculated for each spatial frequency and 
the data was fit with a logistic psychometric function. The subject performed 
significantly above chance for all trials down to 2.21 logMAR. At the critical 



294 A. Horsager and I. Fine 

sampling frequency, each black and white bar falls directly on one row of elec- 
trodes. The resolution was, therefore, directly limited by the spacing of the 
electrodes. 

Taken together, these data suggest that the visual resolution of a blind patient 
implanted with a SSMP A16 retinal prosthesis is limited only by the spacing of the 
electrodes in the array. 



14.6 Models to Guide Electrical Stimulation Protocols 

Achieving useful percepts via electrical stimulation requires satisfying a variety 
of safety and engineering constraints. Useful percepts will require stimulation at 
frequencies higher than subjects' perception of visible flicker (frequencies above 
the "critical flicker frequency"). Second, safety concerns dictate relatively stringent 
charge density limits, since high charge densities have the potential to compromise 
the integrity of electrode material [12, 13] and cause damage to stimulated neural 
cells [48, 49, 67]. Third, the maximum current amplitude that can be produced may 
in some cases be limited by the compliance voltage of the stimulator. A final set of 
constraints include limits in the amount of power available to the implant given the need 
for a long battery life, and power limits inherent in transmitting power inductively, 
resulting in a need to minimize overall charge. 

The models described in this chapter provides an example of how the optimal 
stimulation pattern needed to produce a percept of a given brightness level can be 
determined given a set of constraints. A particular example is given in Fig. 14.14, 
which shows example predictions of threshold current amplitude (graph a), charge 
density (graph b), and overall charge (graph c) for a 500 ms pulse train presented 
on an electrode of typical sensitivity across a range of pulse widths and frequencies. 
The dashed lines represent examples of safety and engineering constraints that 
might restrict the potential set of stimulation patterns. In the example shown here, 
a current amplitude limit of 200 uA, and a charge density limit of 0.35mC/cm 2 . 
Given these example constraints, our model predicts that the most charge efficient 
stimulation pattern, for the conditions and prosthetic device tested here, is a 50 Hz 
pulse train consisting of 0.089 ms pulses. Similarly, for a given compliance voltage, 
the most efficient operation (in terms of energy delivered to the electrodes vs. 
energy dissipated in the current regulator) is when the voltage drop across the elec- 
trodes is near this compliance voltage. These engineering constraints may result in 
the most efficient pulse being at the highest current that can be supplied, making it 
advantageous to manipulate brightness using either frequency or pulse width. These 
models can also be used to calculate the most energy efficient pulse width (chro- 
naxie) [17, 26]. Depending on the assumed constraints of the stimulation protocol, 
models such as these can estimate the best stimulation protocol. 

Of course this ability to evaluate engineering and safety trade-offs across dif- 
ferent pulse patterns need not be restricted to the simple stimulation patterns used 
in this example. Our hope is that this model (or similar models) can be generalized 
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Fig. 14.14 Efficiency predictions for a 500 ms pulse train. In each panel the x-axis represents pulse 
width on a logarithmic axis, and the y-axis represents frequency. Red dashed lines represent a 
current amplitude limit of 200 uA, yellow dashed lines represent the constraint that stimulation 
must occur above the critical flicker frequency of 50 Hz, and blue dashed lines represent the 
constraint of a charge density limit of 0.35mC/cm 2 . Light shading represents pulse widths and 
frequencies that fall outside these constraints. The z-axis represents current (a), charge density 
(b), and overall charge across the entire pulse train (c). Given these example constraints, our model 
predicts that the most charge efficient stimulation pattern is a 50 Hz pulse train consisting of 
0.089 ms pulses, as shown by an asterisk in (c). (Please see online version for full-color representation). 
Reprinted from [33], with permission 



to describe percepts over a wide range of brightness levels, across multiple electrodes. 
It is also to be hoped that models such of these will generalize to other devices, 
though it is of course quite likely that the models needed to explain subretinal 
stimulation will differ substantially from those developed to explain epiretinal stimu- 
lation. While models such as these may always be a crude approximation of 
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perceptual effects, without them developing stimulation protocol procedures will 
remain an ad hoc procedure of trial and error. 



14.7 Conclusions 

The possibility of restoring sight through electrical stimulation has captured the 
interest of laymen and scientists for many years [11, 22, 24, 35, 37, 42, 44, 45, 50, 
53, 54, 61, 62, 75, 77]. As we make progress, the goals associated with restoring 
sight in blind patients (as well as our appreciation of the difficulties that must be 
overcome to reach that goal) have become more sophisticated. In more recent studies, 
the effort has not been simply to create phosphenes, but to create images that are 
predictable over both space and time. 

Over the last 5 years it has become apparent that maintaining close proximity 
between the electrode array and the retinal surface will be critical in developing a 
successful retinal implant. In addition to affecting threshold, separation between the 
array and the retina is likely to compromise the ability to produce small localized 
percepts. As thinner electrode array structures are developed and improved methods 
are developed for attaching the array to the to the retina [29, 71] it should be possible 
to maintain electrodes that remain flush with retinal tissue for indefinite periods, 
resulting in impedance and threshold values that are stable over time. A successful 
prosthesis will require arrays which are stable on the retina, map to predictable loca- 
tions in space, and are of high enough resolution to provide the quality of visual 
information needed to perform useful real world tasks. With the use of electrode 
arrays that meet these criteria, it is likely that the influence of other factors such as 
progression of retinal degeneration and subject age will become more apparent both 
within threshold measurements, and within more complex measures of perception. 

Over the last 5 years, work by a number of groups including ours has demon- 
strated that simple visual percepts generated by direct retinal electrical stimula- 
tion on a single electrode can be modeled relatively simply. This is true both for 
amplitude coding [28] and for manipulations of pulse timing within an electrode 
(frequency encoding). 

Of course a wide variety of challenges remain, even in understanding the effects 
of stimulating a single electrode. For example, apparent brightness is not the only 
perceptual quality that needs to be considered. It is possible that different temporal 
patterns stimulate slightly different subpopulations of neurons, resulting in distinct 
percepts. Moreover, the experiments described here only considered pulse trains or 
stimulation periods of relatively short duration (a maximum of a few seconds). 
Longer periods of continuous stimulation (minutes or hours) may result in long- 
term adaptation, sensitization, and/or retinal rewiring. It is quite likely that frequent 
electrical stimulation over a time scale of weeks and months may result in changes 
in retinal connectivity and responsivity [46]. 

More importantly, it is of critical importance to better understand how neighboring 
electrodes interact in the spatiotemporal domain. The models described in Sects. 14.3 
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and 14.4 simply predict sensitivity at the single electrode level, the extension of models 
such as ours to the spatial domain is an obvious next step. 

While preliminary results with "natural tasks" show promise, it is still not 
entirely clear what kind of spatial resolution is mediated by current prosthetic 
devices. One concern is that, to date, most (though not all) data have come from 
constrained two or four alternative forced choice tasks with training and/or feed- 
back so the subject is performing a constrained discrimination task, not an identi- 
fication task. A subject may be able to discriminate "horizontal" from "vertical" 
without the horizontal line appearing as a horizontal line, and the vertical line 
appearing as a vertical line - all that is necessary is for the two stimuli to be per- 
ceptually distinct. A second concern is the extent of variability across subjects - to 
date no group has reported successful performance across a wide range of tasks 
within all (or even a majority) of implanted subjects. A third concern is that there 
is still some doubt as to whether all electrodes in these arrays map neatly to the 
expected perceptual location in space. As described in this chapter, progress over 
the last 5 years has been rapid, and progress over the next five is likely to bring us 
still closer to a useful prosthetic array. While it is unlikely that we will be able to 
build devices that resemble "natural" vision in the next 5 years, it is possible that, 
even if with some "jumbling" of the sensory input (as is found in cochlear implants 
for hearing) the brain will learn to understand the new sensory representation 
(analogous to learning to interpret modern art sketches for people with normal 
vision). However, we will probably not know the full capacity of the human visual 
system to adapt to make use of retinal implants until these devices are in more com- 
mon use. 
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Chapter 15 

Findings from Chronic Optic Nerve 

and Cortical Stimulation 



Edward M. Schmidt 



Abstract This chapter reviews the experiments that have produced visual sensations 
in humans through electrical stimulation of the central nervous system. Initially, 
surface stimulation of the visual cortex, provided insight into how electrical stimu- 
lation of VI could possibly provide a visual prosthesis for the blind. Intracortical 
microstimulation was then investigated that would allow lower power stimulation 
and increased density of microelectrodes. The stimulation of the optic nerve has 
also been investigated as a possible site for a visual prosthesis. 

The next section is dedicated to what is known and what needs to be done for 
the development of a visual prosthesis. 

The following section examines current research efforts directed towards the 
development of a visual prosthesis. They include optic nerve stimulation, cortical 
surface stimulation and intracortical stimulation of visual cortex. The CORTIVIS 
Program is a comprehensive development of an intracortical visual prosthesis. The 
lateral geniculate nucleus is also being studied as a site for a visual prosthesis. 

The final section of this chapter deals with the developments that are needed for 
a functional visual prosthesis. They include microelectrode arrays, stimulation 
hardware, and low power image sensing and processing circuitry that can control 
the stimulators. 



Abbreviations 


2D 


Two dimensional 


3D 


Three dimensional 


EIC 


EIC laboratories 


HMRI 


Huntington Medical Research Institute 


ICMS 


Intracortical microstimulation 


IIT 


Illinois Institute of Technology 


LGN 


Lateral geniculate nucleus 



E.M. Schmidt (El) 

National Institutes of Health (retired) 

e-mail: emschmidt@atlanticbb.net 



G. Dagnelie (ed.), Visual Prosthetics: Physiology, Bioengineering, Rehabilitation, 301 

DOI 10.1007/978-l-4419-0754-7_15, ©Springer Science+Business Media, LLC 2011 



302 E.M. Schmidt 

MIPS Multimode digital image sensor 

MIT Massachusetts Institute of Technology 

NIH National Institutes of Health 

NY New York 

UC University of Chicago 



15.1 Background 

Visual sensations produced by stimulation of the visual cortex in human patients 
were well known to German neurosurgeons, Kraus [34] and Foerster [27], as early 
as 1924. A number of reports has been published over the years describing the 
effects of electrical stimulation of the visual cortex in lightly anesthetized surgical 
patients [38, 39]. When their visual cortex was stimulated, patients usually report 
small spots of light called phosphenes. 

Shaw [45] obtained a patent for a "Method and Means for Aiding the Blind". In his 
system, a photoelectric tube controlled the intensity and/or frequency of an electrical 
stimulus that was applied directly by internal electrodes, or indirectly by external elec- 
trodes to the visual areas of the brain. Although this appears to be one of the first con- 
cepts of a visual prosthesis, actual implementation of the system has not been found. 

Button and Putnam [11] demonstrated, in blind subjects, visual responses to 
intracortical stimulation controlled by a photoelectric cell. This allowed the sub- 
jects to identify a light source by orientation of the cell. Of the three subjects, one 
was able to follow a flashlight carried by an attendant 15 ft away. 



15.2 Cortical Surface Stimulation 

The first chronic experiment to determine the effects of stimulating the visual cor- 
tex was carried out by Brindley and Lewin [9]. They implanted an array of 80 
electrodes on the medial surface of the occipital pole in a 52 year-old woman who 
had been totally blind for 6 months. The electrodes were platinum squares 0.8 mm 
on a side. They were connected to 80 radio receivers mounted to the skull, beneath 
the pericranium. Alternate receivers were tuned to 6.0 or 9.5 MHz. Pressing a trans- 
mitter coil on the scalp above a receiver and applying the proper frequency pro- 
vided stimulation currents to the associated electrode. With the technology available 
at the time, 80 receivers covered half of the cranium. 

When electrodes that produced phosphenes within 10° of the fovea were stimu- 
lated, the patient reported a very small spot of light, or phosphene, and described it 
as "the size of a grain of sago at arm's length" or "like a star in the sky". Phosphenes 
further from the fovea were sometimes elongated, "like a grain of rice at arm's 
length". The most peripheral phosphenes were round like a cloud. There were three 
electrodes that produced a pair of phosphenes about a degree apart and two electrodes 
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that produced a row of three phosphenes each about a degree apart from the next. 
When multiple phosphenes occurred, stimulus amplitude could not be adjusted to 
produce single phosphenes. For 13 electrodes, weak stimulation produced a single 
phosphene but higher-level stimulation produced a second phosphene in a different 
part of the visual field. 

Other significant findings from this patient were: 

1 . Phosphenes always flickered regardless of stimulation parameters. 

2. Phosphenes moved with eye movement. 

3. Phosphenes could usually be resolved that were produced by electrodes spaced 
2.4 mm apart. 

4. Phosphenes usually ceased immediately at the end of stimulation, but after strong 
stimulation they could persist for up to 2 min. 

5. Stimulation of multiple electrodes could produce simple patterns. 

By improving the experimental prototype, Brindley and Lewin [9] believed that 
at least 200 electrodes per hemisphere could be implanted and would permit blind 
patients to read and navigate. 

Dobelle and Mladejovsky [22] were able to conduct a series of acute experi- 
ments involving volunteers undergoing neurosurgical procedures for removal of 
tumors or other lesions to verify the results of Brindley and investigate the possibil- 
ity of producing a visual prosthesis. Dobelle's data are based on 16 experiments in 
15 volunteers. They were able to confirm most of Brindley's results from a single 
volunteer. A summary of the results obtained from Dobelle's experiments were: 

1 . Phosphene chromatic effects or flicker may or may not occur. 

2. Phosphenes moved with eye movement. 

3. Two-point discrimination was about 3 mm. 

4. Phosphenes appear immediately when stimulation is begun and end immediately 
upon cessation of stimulation. 

5. Phosphenes fade after 10-15 s of continuous stimulation. 

6. Multiple phosphenes are co-planar. 

7. Thresholds ranged between 1 and 5 mA, with 3 mA being typical. 

8. Electrodes of 1, 3, and 9 mm 2 size had similar thresholds and percepts. 

9. Brightness modulation can be achieved by changing pulse amplitude. 

From these studies, it was apparent that to provide a blind person with a stable 
image, either the subject had to learn to use head movements instead of eye movements, 
or the camera used by the visual prosthesis had to move with eye movement. Also, long 
stimulation trains had to be interrupted to compensate for phosphene fading. 

Dobelle's group chronically implanted four volunteers in the 1970s with a sub- 
dural 64-electrode array placed on the medial surface of the visual cortex of the 
right occipital lobe. The wires were terminated in a 72-pin micro-miniature connector 
encapsulated in a transcutaneous pyrolytic carbon pedestal, attached to the cranium 
by platinum bone screws. 

Of these four volunteers, two had useful results for the future of artificial vision. 
One of them, blind for 10 years and implanted in 1975 at age 33, could perceive 46 
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useful phosphenes out of 60. Using six phosphenes with a layout similar to that of 
a Braille cell, he could read cortical Braille at approximately five words per min but 
he could only read tactile Braille at one word per min [23]. He could identify the 
orientation of white strips of tape on a blackboard by manipulating a video camera 
mounted on a joystick. His phosphene map stayed constant and his thresholds only 
had small changes over 10 years. 

Another volunteer, blind for 7 years and implanted in 1978 at age 41 with an 
identical 64-electrode array could perceive 21 useful phosphenes. Over the last 25 
years, his phosphene map and thresholds have stayed constant. In the late 1990s this 
volunteer benefited from the miniaturization of electronic components and advances 
in computer technology. He was the first blind volunteer to wear a miniature video 
camera mounted on his eyeglasses and a sub-notebook computer, a stimulator, and 
batteries in a waist pack [24]. Using an edge detection algorithm, the images from 
the video camera were processed by the computer, which selected the electrodes that 
produced phosphenes on or near the high-contrast areas of the images. The stimulator 
in turn generated the proper stimuli for the selected electrodes. 

Compactness and portability of the system allowed the subject to detect and 
negotiate objects, follow a child walking slowly and close to him in a hallway, 
follow a strip of black tape on the floor, enter a room, grab a ski cap hung on the 
opposite wall, turn around, walk towards a mannequin and put the cap on its head. 
Accompanied by staff in the NY City subway system, an environment he was 
familiar with, he could get inside a subway car. He found it easier to differentiate 
the space between two cars and an open car door with his visual prosthesis than 
with his cane. 

The results of the research done on these two volunteers, particularly the last 
one, were quite promising. If they could achieve all this using a single array with a 
limited number of phosphenes, the logical conclusion was that with two arrays, 
blind patients would have more phosphenes, creating images with higher resolu- 
tion, therefore giving them more independence and mobility. 



15.3 Intracortical Microstimulation 

In cat motorsensory cortex, Stoney et al. [46] showed that thresholds for facilitation 
of spinal motorneuron pools by intracortical microstimulation (ICMS) could be 
as low as 2 uA, which is 1/100 of the threshold for producing similar effects with 
surface stimulation (Asanuma et al. [2]). These results led Dobelle & Mladejovsky 
to try ICMS in patients where the cortex was going to be surgically removed. 
This was not successful, possibly due to pathological involvement of the cortex in 
question [22]. 

In 1980 Bartlett and Doty [4] investigated the ability of primates to detect ICMS 
of the visual cortex. They advanced microelectrodes through the visual cortex and 
recorded the primate's threshold for detection of the stimulus. They found thresh- 
olds significantly lower than surface stimulation, with some thresholds as low as 
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2 uA (0.2 ms at 50 Hz). It was not apparent if the primates were responding to 
phosphenes similar to those produced by surface stimulation in humans. If the pri- 
mates were seeing phosphenes then it appeared that it might be possible to produce 
an intracortical visual prosthesis requiring much less power than using surface 
stimulation. This question could only be answered in human subjects. 

Dr. Hambrecht, who was Director of the Neural Prosthesis Program at the 
National Institutes of Health (NIH), assembled a team of scientists to determine if 
ICMS was suitable for use in a human visual prosthesis. Protocols were approved 
at the NIH and at the University of Western Ontario to test patients who were under- 
going surgery for excision of epileptic foci in the visual cortex. Three patients were 
studied in Canada for lh each [3] by first briefly stimulating the exposed cortex 
with a surface electrode and then inserting pairs of electrodes into the region where 
the patient reported phosphenes. As the electrodes were advanced through the cortex, 
the threshold for phosphene production dropped from as high as 5 mA at the sur- 
face to about 20 |iA at 2-3 mm from the surface. Near threshold, the phosphenes 
were usually blue, yellow or red. The phosphenes did not flicker. With interleaved 
stimulation of two microelectrodes that were 0.7-1 mm apart, the patient reported 
"two blobs fusing." When the tip separation was 0.3 mm, the percept was a singular 
round shape. 

The next step in developing a visual prosthesis was to chronically implant a 
blind human volunteer with an array of intracortical electrodes. Hambrecht [29] 
provided an excellent review of the next study and Schmidt et al. [44] provided the 
details of the human experiment. This study was limited to a 4-month investigation 
as set out in the approved protocol. 

Thirty-eight microelectrodes were implanted in the visual cortex. They consisted 
of 12 single microelectrodes and 18 pairs. The spacing between pairs of microelec- 
trodes was 250, 500 or 750 |im. Two of the microelectrode leads were broken at the 
time of implantation and only two of the remaining 36 microelectrodes failed to 
produce phosphenes. Due to the untimely breakage of a number of microelectrode 
wires, planned pattern recognition studies could not be conducted. 

The phosphenes produced by ICMS were similar to those reported in the 
Canadian study [3]. A summary of the results obtained with ICMS were: 

1 . Phosphenes never flickered. 

2. Phosphenes moved with eye movement and a group of phosphenes maintained 
their relative positions with eye movement. 

3. Stimulation of microelectrodes, with tips separated by 0.5 mm, produced sepa- 
rate phosphenes. 

4. Phosphenes appeared immediately after the beginning of stimulation and except 
for rare occasions, disappeared at the termination of stimulation. 

5. When stimulation continued beyond a second, phosphenes usually 
disappeared. 

6. By interrupting a long stimulation pulse train with brief pauses, the duration of 
phosphene perceptions could be increased. 

7. Multiple phosphenes were co-planar. 
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8. Threshold currents were as low as 1.9 uA, while most of the microelectrodes 
had thresholds below 25 uA. 

9. Thresholds for cathodic-first stimulation with a biphasic pulses was always less 
than anodic-first. 

10. Varying the stimulus frequency, pulse width or amplitude modulated phosphene 
brightness. 

1 1 . Near threshold stimulation, the phosphenes were often reported to have colors 
of red, blue or yellow, but never green. When the stimulation levels were 
increased, the phosphenes generally became white, grayish or yellowish. 

12. Brighter phosphenes could obscure dimmer phosphenes. The subject had to 
adjust current levels so that all phosphenes in a group could be seen at the 
same time. 

13. Stimulation of some microelectrodes produced a second closely spaced phos- 
phene at a higher current than the first. When three microelectrodes that produced 
two phosphenes were simultaneously stimulated, producing six phosphenes, they 
appeared in almost a vertical row and the subject identified them as a letter "I". 

As specified in the approved protocol, at the end of the 4-month implantation 
period the electrode lead wires were removed, along with several of the microelec- 
trodes, for examination. The volunteer never experienced residual side effects from 
the implant. Two years after the study was completed, she suddenly died. An 
autopsy revealed that she had a ruptured berry aneurysm located in the hemisphere 
opposite from where the microelectrodes had been implanted. The conclusion, as 
determined by an investigative panel, was that the experimental visual implant was 
not responsible for her death. 

Although a limited amount of information was obtained from this volunteer, an 
intracortical visual prosthesis looks promising. 



15.4 Optic Nerve Stimulation 

The next feasible site for a visual prosthesis, after the retina, is the optic nerve. Prior 
to consideration of this site, considerable work was done on cats in the development 
of a spiral nerve cuff electrode [51]. Selective recruitment of different muscles 
innervated by the sciatic nerve could be accomplished by electrode selection within 
the cuff [28]. With this background, Veraart decided that a visual prosthesis based 
on optic nerve stimulation might be feasible. 

The first optic nerve implant volunteer was a 59-year-old female that was totally 
blind due to retinitis pigmentosa [52]. The self-sizing cuff electrode contained four 
contacts. Electrical stimulation of the electrodes never produced any sensation 
other than vision. With each stimulation, using a given set of parameters, the patient 
reported multiple phosphenes in a cluster of 2-5, or arranged in rows, or clumps of 
6-30. Stimulus currents as low as 30 uA were capable of eliciting phosphenes. 
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Although changing the stimulation parameters generated a large number of different 
phosphenes, the actual parameters were not reported with the retinotopic stimula- 
tion map 

Pattern recognition studies were undertaken when the subject used a head- 
mounted video camera [53]. Either 4 or 24 phosphene locations were used in the 
tests. When the video image intersected one or more phosphene locations, a 
sequential pattern of stimulation was produced. Forty-five simple patterns were 
presented and after learning, the subject reached a recognition score of 63% with a 
processing time of 60 s. This study shows that information can be delivered through 
optic nerve stimulation, but at an extremely slow rate. 

One of the last reported studies with this subject involved object localization, 
discrimination, and grasping [25]. After training, the subject reached 100% success 
rate in performing all three tasks. Localization was achieved in 20 s while discrimi- 
nation required 40 s. Grasping required no more than 6 s. This study provides data 
that indicates optic nerve stimulation might be useful in daily life if more phos- 
phenes can be generated simultaneously rather that sequentially. 

A second blind volunteer was implanted, but only the details of the surgery have 
been reported [8]. Unfortunately, do to the retirement of Dr. Veraart, the optic nerve 
visual prosthesis program in Belgium may not continue. The implanted patients 
will however be followed by Dr. Delbeke. 

For an optic nerve visual prosthesis to be useful, many more stimulations sites 
are required than can be obtained with cuff electrodes. The Utah slanted electrode 
array (USEA) [7] could theoretically provide up to 100 independent phosphenes. 
This assumes that the electrode array can be implanted in the optic nerve and each 
electrode produces an independent phosphene. For peripheral nerve, a pneumati- 
cally actuated impact insertion tool was developed [42]. How such a device can be 
used for optic nerve implantation remains to be determined. When one solves 
the electrode array implantation problem, the next question is how many electrodes 
are required to provide a useful optic nerve visual prosthesis. 



15.5 What Is Known and What Needs to Be Done 

From the results of optic nerve and visual cortex stimulation, the only type of visual 
prosthesis that we can consider for a blind patient, at this time, is a scoreboard 
type of display. Implanted subjects have shown that meaningful information can be 
obtained with 20-109 phosphenes, if they remain stable over time. The image 
would be a 2D set of phosphenes. 

Biologically safe stimulation parameters for the different types of electrodes 
used in a visual prosthesis have to be determined. 

Gray scale rendition can be obtained by varying stimulation parameters, but 
this depends on knowing the threshold current of each electrode. A stimulus- 
brightness curve for each electrode would have to be obtained in order to properly 
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scale stimulation intensity for a given scene. If a very bright phosphene is near a dim 
phosphene, the dim phosphene may not be observed. A stimulus-brightness curve 
can be generated from a patient's verbal response, or using a reaction time task to 
the presentation of different stimulus intensities. Brighter phosphenes are observed 
sooner than dimmer phosphenes. To collect this data is a formidable task, but may 
prove useful with the initial implant patients. Most likely it will be an indispens- 
able aspect of exploring the potential and limitations of all visual prostheses. 

Colored phosphenes occur near threshold currents with no guarantee as to what 
color will be produced. Threshold currents will be different for the electrodes in the 
array and must be determined for each electrode. At this time it is doubtful if a 
meaningful color visual prosthesis can be developed. 

With long trains of stimulation, phosphenes will fade. If brief pauses are pro- 
vided in the stimulus train, the intensity can be maintained for a longer period. 
A suggested mode of operation for a long stimulus train is to stimulate for 320 ms 
and then pause for 32 ms before repeating the stimulation pulses. This strategy will 
have to be verified in patients. 

When using arrays with a large number of electrodes, we are not sure where the 
phosphenes will appear on the subject's visual map. A rapid phosphene mapping 
technique is required so that the correct transformation of the camera scene to 
appropriate electrodes can be made. 

Phosphenes move with eye movement so that if a patient looks at a phosphene it 
will move away. In order to center the object in the field of vision of the prosthesis, 
the patient will have to make compensating head movements if the camera that is 
sensing the scene, is mounted on an eyeglass frame. One approach to this problem 
is to develop a miniature camera that tracks eye movement and moves in the same 
manner. Another approach is to implant a miniature camera in the eye. 

After an image is obtained from a video camera, the data needs to be processed 
before stimulus signals are applied to the appropriate electrodes. This can be as 
simple as threshold detection and any signal above threshold initiates a constant 
level of stimulation. Edge detection algorithms can be employed to minimize the 
number of electrodes that are stimulated. When reading black on white text, con- 
verting the positive image to a negative one would produce white letters on a black 
background, again reducing the number of electrodes stimulated. 

To summarize what is needed for a visual prosthesis implant: 

Electrochemically safe electrode arrays 

Biocompatibility of implants 

Means of efficiently obtaining electrode threshold current 

Threshold current stability 

Means of efficiently mapping phosphene location 

Determine phosphene brightness versus current 

Determine length of stimulation and duration of pauses to stabilize brightness 

Develop a camera coupled to eye movement 

Image processing 
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15.6 Current Research Efforts 

The research on a visual prosthesis using sites other than the retina can be divided 
into a number of sub categories. They are optic nerve stimulation, lateral geniculate, 
surface stimulation of visual cortex, intracortical stimulation of visual cortex, stimu- 
lation hardware, microelectrode arrays, miniature cameras and animal models. The 
ongoing research in each of these areas will be listed separately. 



15.6.1 Optic Nerve Stimulation 

Ren and co-workers at the Shanghai Jiao-Tong University in Shanghai, China has 
established a program called C-Sight to investigate implantation of penetrating 
microelectrodes in the optic nerve for a visual prosthesis [41]. They are investigat- 
ing an image acquisition and processing system, a data telemetry system, a neural 
stimulator, and an implantable micro-camera system for an optic nerve visual 
prosthesis. 

Another approach that is being actively studied in Germany is the use of regen- 
eration microelectrode arrays. These electrodes consist of a wafer that has a number 
of holes into which nerve fibers can regenerate. The holes contain electrical con- 
tacts that enable single or a few nerve fibers to be stimulated. The optic nerve is cut 
and sutured to either side of the perforated microelectrode array. In rats, recovery of 
visual evoked potentials occurred in 2-8 weeks [30]. If regeneration through the 
perforated microelectrode arrays can be successful in primates and chronic stimulation 
of fibers can be shown to produce phosphenes then one could consider implanting 
this type of microelectrode in humans. One of the disadvantages of this type of 
microelectrode is that the optic nerve has to be cut and success of the implantation 
cannot be known for weeks or months. This might discourage some volunteers. 

A group at Osaka University, Japan is investigating a different approach by 
stimulating the fibers in the optic nerve head inside the eye [26]. The advantages of 
this approach over the optic nerve cuff are that the exposure of fibers across the rim of 
the optic nerve head allows stimulation of small groups of fibers, and the intraocular 
surgical procedure is less invasive. 



15.6.2 Cortical Surface Stimulation 

After Dr. Dobelle died, his family donated the project, his patent and the technology 
to SUNY (State University of New York) at Stony Brook, in May 2006. Members 
from the staff at SUNY and Avery Biomedical Devices have teamed up to 
completely redesign the system used with the 16 patients implanted in Portugal. 
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The redesign of the electrode array and electronics package will be completed 
before seeking FDA approval to implant patients in the USA. 

Chowdhury and colleagues, in Australia, has been investigating a cat model for 
evaluating prototype cortical surface electrode arrays for a visual prosthesis [16, 17]. 
At present, this group does not have any plans for implanting human subjects. 



15.6.3 Intracortical Stimulation of Visual Cortex 

Because the National Institute of Health (NIH) is funded year to year by Congress, 
long-term patient care cannot be guaranteed. Thus the NIH administration decided 
not to continue the Visual Prosthesis Program for that reason. The scientists in the 
program were given the task of finding an appropriate University hospital that had 
access to the engineering expertise needed to carry out the Visual Prosthesis 
Program. Troyk and co-workers, at the Illinois Institute of Technology (IIT), formed 
a consortium consisting of IIT, University of Chicago (UC) and their Medical 
Center, EIC, and Huntington Medical Research Institute (HMRI). The NIH technol- 
ogy was transferred to IIT. The role of IIT is to develop implantable microelectrode 
arrays that contain RF powered and controlled stimulator packages and establish 
safe stimulation parameters for the microelectrodes [49]. EIC provides the electro- 
chemistry expertise to properly develop iridium oxide stimulating electrodes [50]. 
The University of Chicago conducts the primate psychophysics experiments [6] 
and the Medical Center implants the primates in preparation for human implants. 
HMRI conducts safety experiments and histological evaluations of all implants. The 
Wilmer Eye Institute at Johns Hopkins University has been added to the consortium 
for evaluating human implants. A human implant is envisioned within 2 years 

The University of Utah has conducted a number of studies aimed at implanting 
microelectrodes in the visual cortex. Normann and co-workers have developed a 
micro-machined electrode array consisting of 100 microelectrodes [37]. His group 
has conducted a number of studies that could lead to a human implant in the near 
future. They have looked at the histological effects of implanting these electrodes 
[35], the results of acute implantation in human neocortex [31] and the thermal 
impact of active arrays implanted in the brain [33]. They envision a intracortical 
visual prosthesis system employing 625 microelectrodes. The system receives 
video information from a micro-camera mounted in eyeglasses, processes the 
images with a computer and transmits the information over a telemetry system to 
stimulators on the electrode arrays. 



15.6.4 CORTIVIS Program 

A consortium of European Research Institutions has formed under the coordination of 
Dr. Fernandez in Alicante, Spain, called CORTIVIS [18]. The aim of the consortium 
is to develop a visual prosthesis based on intracortical stimulation of the visual 
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cortex. They have developed an image processing system that mimics the human 
retina. The signals from the retina module are converted into neuromorphic pulse- 
coded signals through a circuit that emulates the function of the retinal ganglion 
cells [1]. They are currently performing in vitro experiments (for biocompatibility, 
in vivo animal experiments (acute and chronic)) and working towards human 
implants. Initially they will use the Utah Electrode Array [37] while developing a 
3D probe array. 



15.6.5 Lateral Geniculate Stimulation 

Pezaris and Reid at Harvard Medical School [40] have demonstrated in primates 
that microstimulation in the lateral geniculate nucleus (LGN), which is the relay 
between the retina and the visual cortex, produced localized visual percepts. To 
assess the effects of microstimulation of LGN in a primate, an eye movement task 
was used with visual targets presented on a computer screen or through microstimu- 
lation. Saccades made to electrical targets were comparable to saccades made to 
optical targets. They estimate that 200-300 stimulation sites are available in the LGN. 
This would be adequate for reading with a visual prosthesis. However, developing the 
required electrode arrays and implanting them in the LGN is a formidable task. 



15.7 Microelectrode Arrays and Stimulation Hardware 

The University of Michigan has a long history in the development of multi-site silicon 
stimulating probes [54, 55]. Their resent development is a 64-site wireless micro- 
stimulator (Interstim-2B) [36]. Up to 32 chips can be connected in parallel to drive 
2,048 stimulation sites. This should be more than adequate for any currently 
planned visual prosthesis. 

PolySTIM Neurotechnologies Laboratory in Montreal, Canada, Has developed 
a power efficient stimulator for an intracortical visual prosthesis [19]. 

Delbeke et al. [20] have developed a microsystem based stimulator for an optic 
nerve prosthesis. 



15.7.1 Miniature Cameras 

A group at Shanghai Jiao Tong University, Shanghai, China have developed a 
micro-camera that can be implanted in eye and powered by a solar array positioned 
in front of the iris [12]. Since phosphenes move with eye movement, an eye- 
mounted camera should help to stabilize the perceived image. The camera provides 
a 32x32 element image, which with their simulation studies allowed a subject to 
recognize simple scenes. Through simulations, they also found that a 12 x 12 array 
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of pixels was sufficient to recognize Chinese characters [13]. With a lOx 10 array, 
the recognition level dropped to slightly under 50%. 

The retinal visual prosthesis group at the University of Southern California, 
USA, is also developing a camera implantable in the eye. 

PolySTIM Neurotechnologies Laboratory has developed a CMOS multimode 
digital image pixel sensor (MIPS) for a visual prosthesis [43]. Three selectable 
operation modes are combined in the proposed MIPS: a high dynamic range loga- 
rithmic mode, a linear integration mode, and a novel differential mode between two 
consecutive images. This last mode allows 3D information for a cortical stimulator. 



15. 7.2 Animal Models 

The major groups that are investigating the entire realm of aspects leading to human 
implants of a visual prosthesis are employing animal models at some stage of their 
work. Other groups are just looking at animal models and how they might apply to 
a visual prosthesis. 

The group at IIT/UC [6] have chronically implanted arrays of microelectrodes in 
non-human primates to evaluate intracortical stimulation. One of the major findings 
was that the stimulation package originally developed under an NIH contract, as 
described on the IIT web site [32], could not be connected to the intended number of 
microelectrodes at surgery. Small electrode-stimulator modules had to be developed 
that used telemetry to transmit power and stimulation in formation. At MIT, 
Tehovnik and colleges [47, 48] have used moveable microelectrodes to map the 
generation of saccadic eye movements and study how these data might be appli- 
cable to a visual prosthesis. DeYoe [21] and Bartlett [5] at the University of 
Rochester used moveable microelectrodes to study stimulation parameters and 
laminar distribution of phosphene production in non-human primates. These studies 
will aid in the development of a human visual prosthesis. 



15. 7.3 Image Processing and Phosphene Mapping 

Part of the CORTIVIS project is the development of a bio-inspired visual processing 
front-end that would be placed between the photosensor array and the stimulator 
for an intracortical visual prosthesis [18]. The images are processed by a set of 
separate spatial and temporal filters that mimic the functions of the photoreceptors, 
amacrine and bipolar cells in order to enhance specific features of the captured 
visual image. 

The C-Sight Visual Prosthesis Group in China has been studying tactile phos- 
phene mapping in sighted subjects using a head mounted display for the simulated 
phosphenes and a 19 in. touch screen to record the subject's tactile position [14, 15]. 

PolySTIM Neurotechnologies Laboratory has surveyed image processing strategies 
that can be used with a visual prosthesis [10]. 
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15.8 Conclusion 

With the wide range of research that is currently underway to develop a visual prosthesis 
it is possible that we will see several groups implanting humans in the next 5 years. 
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Part IV 

Towards Prosthetic Vision: Simulation, 

Assessment, Rehabilitation 



Chapter 16 

Simulations of Prosthetic Vision 

Michael P. Barry and Gislin Dagnelie 



Abstract Simulations of prosthetic vision can provide requirements and specifications 
for prosthesis designs and stimulus conditions; these requirements are expected to 
differ according to the visual task. Studies reviewed here include examinations of 
visual acuity, reading, face and object recognition, hand-eye coordination, way 
finding, visual tracking, and simple design feasibility. Based on these studies, 
visual acuity with prosthetic vision seems to depend most on the resolution of 
perceived phosphenes. Given usable visual acuity, all visual tasks that have been 
evaluated in simulations with variable dot counts demonstrate some significant 
dependence on the number simulated phosphenes provided. Some tasks also have 
more unique dependencies: Facial recognition seems quite sensitive to the number 
of gray levels and the relative size of dots and spacing. Wayfinding is most depen- 
dent on the angle of view captured by the camera. In many of the simulation studies 
practice was found to be an important factor for successful task performance. As 
visual prosthesis development becomes less limited by technological barriers, find- 
ings from simulation studies may become increasingly important for the design of 
implants and rehabilitation programs. 



Abbreviations 

Symbol for minutes of arc 
DBS Deep brain stimulation 

HMD Head-mounted display 

LGN Lateral geniculate nucleus of the thalamus 

logMAR Logarithm of the minimum angle of resolution 
MPDA Multi-photodiode array 
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16.1 Introduction 

Although not perfect models of visual prostheses, simulations of prosthetic vision 
provide insight into how these prostheses can theoretically function. While tests of 
actual prostheses suffer the burdens of device construction and implantation and all 
the associated costs and approvals, simulations are relatively simple to implement 
and require less regulatory oversight. As such, before any sizable clinical trials 
were possible, simulations of visual prostheses were used to investigate the usability 
of prosthetic vision. Now, as clinical trials move forward, prosthetic vision simula- 
tions still provide insight on how different elements of the technology interact and 
affect performance. Simulations of theoretical device designs also help guide devel- 
opers in building next-generation prostheses. 

The first studies utilizing simulations of prosthetic vision were published in 
1992 by a group at the University of Utah [3-5]. Each of these initial studies simu- 
lated simple square grids of dots by covering a small screen (1.7° of visual field 
across) with a film containing chemically etched holes. Using this simulation 
scheme, Cha et al. evaluated normally sighted subjects with tests of visual acuity 
[3], reading [5], and wayfinding [4]. As the availability of technology progressed 
over time, simulations of prosthetic vision evolved to software-based implementa- 
tions of visual prostheses; however the basic categories of tests have persisted, with 
a few additions: face and object recognition [14, 15, 21, 29, 30, 33, 34], hand-eye 
coordination [14, 15, 21, 29], visual tracking [20, 31], and purely computational tests 
[25]; most of these tasks can be implemented and explored in virtual [1,7, 13, 29, 

321 as well as real [4, 13, 15, 29] environments. These simulation studies, taken 
together, provide a wide range of knowledge on what may be possible with actual 
prostheses, what resolution and other device properties may be required for specific 
tasks, and in which directions prosthetic development should proceed. 

In this chapter we will summarize studies that have been performed in these 
different categories. We will open with some general remarks about the ways simu- 
lations are implemented and some of the basic parameters that can be varied. 
For the sake of consistency, the words "array" and "phosphene" will be used only 
to refer to actual prostheses and their associated percepts, while "grid" and "dot" or 
"simulated phosphene" will be used to refer to simulations of visual prostheses. 



16.2 Simulation Techniques and Basic Parameters 

In a typical prosthetic vision simulation, a sighted individual is presented visual stimuli 
that approximate what the visual prosthesis wearer is expected to perceive. Typically, 
these are images in which the original resolution has been reduced to represent the 
stimulating array that is to be implanted in a blind subject, with individual dots or 
squares of light representing the phosphenes elicited at each point of stimulation. 

Figure 16.1 shows the implementation of a prosthetic simulation commonly 
used in our laboratory, where pixelized images are presented in a video headset, 
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virtual scenery 




Fig. 16.1 Schematic arrangement in a typical prosthetic vision simulation. The filtering engine 
(top center unit) converts an incoming video stream from either a real or a virtual scene into 
pixelized imagery. The head-mounted display (HMD) in this arrangement is used to present the 
imagery to the subject and monitor the subject's gaze through a built-in video camera observing 
the pupil. A scene camera mounted on the HMD can be used to provide live video for filtering. 
The pupil-tracking software (top left unit) provides the filtering engine with near-real time gaze 
information, allowing the imagery to be stabilized on the subject's retina, simulating a fixed position 
of the stimulating array 



with either a scene camera on the headset or rendering software under control of a 
gaming engine (HalfLife; Valve Software, Bellevue, WA). Other configurations 
may involve a monitor display, a hand-held or glasses-mounted camera, or other 
image capture and display methods. Central in all simulations is a processor that 
transforms the incoming video stream into an outgoing stream that fulfills the prop- 
erties of prosthetic vision as they are envisioned by the experimenter. 



16.2.1 Gaze Tracking and Image Stabilization 



An important aspect of prosthetic vision with an external (head-worn, hand-held, or 
stand mounted) camera is the loss of the effects of eye movements to which every 
sighted person is accustomed. As illustrated in Fig. 16.2 (left panel) an eye move- 
ment executed by a sighted person makes the image of the object being observed 
shift across the retina, and hence across the projection areas in the visual cortex. 
The visual system deals with this by signaling to the visual cortex that an eye move- 
ment is being made, so the shift of the image is perceived as a stable rather than a 
shifting world. This situation changes dramatically (Fig. 16.2, central panel) if the 
image from a stationary external camera is presented to the visual system in the 
form of electrical impulses from a set of electrodes attached to the retina or higher 
visual centers: An eye movement executed by the prosthesis wearer will still signal 
the visual cortex that an image shift should happen, but since the camera and 
electrodes are stationary no such shift occurs, and the resulting percept is a discon- 
certing jump of the scene. 

Retinal implants that perform image capture directly inside the eye will not have 
this problem, as the image will shift according to eye movements. Of the current 
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Eye movements: 
Natural vision 



Eye movements: Ey e movement compensation: 

Head-mounted camera Simulated retinal implant 




Fig. 16.2 The effect of eye movements on stimulation of the visual system in natural vision (left 
panel) and in prosthetic vision with an external camera, both without (center panel) and with 
(right panel) compensation through gaze tracking 

implants, only the multi-photodiode array (MPDA) of Retina Implant AG provides 
this capability. For all devices with an external camera the situation can be remedied 
by tracking the prosthesis wearer's eye position and presenting a corresponding 
shift of the image to the implant. This would be done most easily by using a wide- 
angle camera and instantly panning the section to be presented to the prosthesis 
wearer in accordance with the current direction of gaze. Such accurate and instan- 
taneous gaze tracking is not currently used, however. 

Accurate prosthetic vision simulations should therefore have the ability to 
mimic gaze stabilization. In the diagram of Fig. 16.1 this is implemented through a 
pupil-tracking video camera built into the HMD, eye-tracking software (Arrington 
Research, Scottsdale, AZ), and a resulting offset of the filtered imagery according 
to the updated gaze position; typically this is done at 30 or 60 frames per second, 
but more rapid systems are now available. 



16.2.2 Filter Engine Parameters 

In order to present imagery that closely resembles what a prosthesis wearer is 
expected to perceive, the filtering engine needs to transform the incoming video 
frames according to a number of important aspects. Roughly, these can be catego- 
rized into four groups: raster spatial properties, dot spatial and temporal properties, 
and dynamic background noise. 



16.2.2.1 Raster Spatial Properties 



Typically, the experimenter will have a specific implant configuration in mind and 
will sub-sample the incoming image to match that configuration. For a retinal 
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implant, the electrode arrangement will most likely be rectangular and regular, 
although hexagonal and/or radially expanding configurations could in principle be 
used, in order to conform more closely to the native properties of retinal processing. 
In all cases the incoming image is reduced in resolution by grouping the intensity 
and color values within the aperture of each prospective dot position. As an example, 
a typical 320x240 pixel camera image can be down-sampled to simulate a 10x6 
implant by dividing it into 60 rectangular subfields of 32 x 40 pixels each, and aver- 
aging the pixel values within each rectangle to yield a single value that will be 
represented by the simulated phosphene. Color information will typically be dis- 
carded, since only grey scale values are thought to be meaningfully conveyed. 

There are several instances where a regular grid of simulated phosphenes is not 
an adequate representation of what the implant recipient is expected to see. Most 
importantly, this is the case for implants beyond the retina. Stimulation of the optic 
nerve, LGN, or primary visual cortex should still provide a predictable phosphene 
array, depending on the accuracy of electrode placement, and these irregularities 
can be built into the simulated phosphene map. 

Even for a retinal implant there may be distortions of the regular grid. In the 
normal retinal anatomy the centermost fovea does not contain any secondary neurons, 
so many neurons at l°-2° eccentricity in the retina will correspond to locations 
much closer to fixation in the visual field, and stimulating those neurons will cause an 
apparent contraction of the image: phosphenes will be denser immediately around 
the point of fixation, and correspondingly sparser in a ring at 2°-4° eccentricity. 
In addition, the retinal rewiring process described in Chap. 3 will cause inner retinal 
neurons to migrate from their original positions, and may thus convey random 
scatter to the perceived phosphene positions. The magnitude of both effects can be 
estimated, but to our knowledge have not been taken into account in simulations 
of a retinal prosthesis. On the other hand the crude resolution of most current pros- 
theses, with electrode separations of approximately 2°, reduces the need for such 
refinements. 

In addition to the overall arrangement of dots in the raster, several parameters 
can specify raster properties: 

• Dot number. This quantity corresponds to the number of electrodes in the 
implant. 

• Dot density: This quantity determines the center-to-center distance between dots, 
and is typically chosen to correspond to the inter-electrode distance of the 
implant. For rectangular grids it is common for density to be equal in the two 
perpendicular directions. Note that density is the inverse of center-to-center 
distance. 

• Dot spacing: When viewing the dot grid one can envisage each dot as being 
situate at the center of a "unit cell," and the dot may or may not fill the entire 
cell. For round dots in a rectangular (rather than square) grating, dot spacing will 
be different in the two orthogonal directions. The space between dots and 
the background intensity light filling that space will be further discussed under 
Sect. 16.2.2.2. 
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Grid size: This quantity has a direct relationship to the previous two; it is 
common for one of the three parameters to be kept constant, and study the trade- 
off between the remaining two. 

Dot drop-out: A subset of electrodes in an implant may prove non-functional 
after implantation or lose functionality over time; this loss of function can be 
caused by either the implant itself or by degeneration of the tissue substrate; to 
model this, a subset of dots may be omitted; typically this subset is chosen at 
random, and not altered while testing a given subject over multiple sessions, to 
investigate whether adaptation may occur to this localized absence of image 
information. 

Effects of several grid parameter changes are shown in Fig. 16.3. 



16.2.2.2 Dot Spatial Properties 

Phosphenes elicited by localized electrical stimulation in blind individuals have 
generally been described as small round dots, varying in size from a pea to a quarter 
at arms length, and either sharp or fuzzy in appearance; some subjects have 
described rings or dark dots on a lighter background, depending on the stimulus 
conditions. This illustrates a basic problem when rendering images in even the 
simplest prosthetic vision simulation. The square pixelization commonly employed 
to hide a person's identity in the media (see Fig. 16.4, left panel) lend themselves 
to rapid image rendering and have been used extensively by one research group 
[16, 17, 24, 26-28], but may not be an optimal representation of what is described 
by patients undergoing stimulation. Other groups have spent considerable effort on 
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Fig. 16.3 Illustration of the effects of grid and dot parameters on the display of a text fragment 
with pillbox-shaped dots. All changes are relative to the "standard condition" in the center of the 
figure. In the top right panel grid size is changed without increasing dot size or number, whereas 
in the bottom left panel the dot number is changed, and in the top left panel dot size is changed 
while keeping the gaps separating the dots equal 
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Fig. 16.4 Examples of pixelization used in prosthetic vision simulations. In both examples a 
rectangular raster was used. The left panel shows a 14 x 14 cell grid with square pixelization of 
a face (courtesy Dr. Wentai Liu), while the right panel shows a 4x4 grid with Gaussian dot profile 
as seen by a subject in our laboratory inspecting the scoop of a spoon 



the creation of model phosphenes with precisely specified spatial properties. 
Generally, the following parameters are specified: 

• Shape: Although most phosphenes seen by patients are not perfectly round, the 
most common shapes used in simulations have been bright circles on a dark 
background, as shown by the examples in Figs. 16.3 and 16.4 (right panel). 

• Profile/size: There is a variety of ways in which the light representing the intensity 
in the scene can be distributed across the unit cell. The most common profiles 
chosen are rectangular and Gaussian; the extent to which the light in one cell 
merges with that in neighboring cells depends on the radius of the pillbox (0.495 
in the example in Fig. 16.5; hence there is no overlap in the right half of the 
figure) or the value of a (four values shown). If a Gaussian profile is chosen 
there is always some overlap, making the use of Gaussians much more compu- 
tationally intensive in a real-time simulation. The increased speed of general 
purpose processors and the use of dedicated hardware have led to more frequent 
use of Gaussian profiles in recent simulations, since they correspond more 
closely to the reports of prosthesis wearers [22] . 

• Intensity/contrast: Most simulations use bright dots and modulate the peak inten- 
sity of the dots to represent local brightness in the scene, on a black background. 
Yet it is unlikely that prosthesis wearers will experience such high contrast per- 
cepts: Patients blind from outer retinal degenerations describe their world as 
grey rather than black. For this reason some simulation studies have explored 
the dependence of subject performance on contrast. In some cases this was 
done by only changing dot brightness but leaving the black background; this 
reduces brightness rather than contrast and has very little effect as the subject 
adapts to the lower light level. Increasing the background intensity, with or with- 
out a reduction in dot intensity, is an appropriate way to reduce contrast. 

Some studies (e.g., Chap. 17) have modulated the radius of pillbox dots rather 
than their intensity, but a systematic comparison of the two methods across 
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Fig. 16.5 Illustration of pillbox and several Gaussian dot profiles. The left half of the figure 
shows the profiles in isolation, while the right half shows the effect of interactions with neighbors 
in a square grid. Four different a values have been chosen to illustrate the effect of neighbor 
interactions. Inter-dot distances and peak intensity values have been normalized. Note that the 
radius of the pillbox equals 0.5 in this example, and that a modest increase in a (0.33-0.5) causes 
dramatic blurring, due to long-range Gaussian overlap 

multiple tasks has not been performed; in principle a similar intensity modulation 
could be used with Gaussian dots, by modulating a rather than peak intensity. 
To our knowledge this has not been attempted. 

• Grey scale/size resolution: Natural vision is capable of resolving subtle differ- 
ences in shading, even in the absence of color, but this is unlikely to be the case in 
prosthetic vision. For this reason a number of simulation studies have examined 
subjects' visual performance under conditions of reduced grey scale resolution. 
Typical resolutions used range from 2 to 16. 

• Homogeneity: Most simulation studies use identically shaped dots, but it is inevi- 
table that phosphenes perceived by prosthesis wearers will vary in intensity, size, 
shape, and other aspects. Until more information from prosthesis wearers is 
obtained it may be premature to build such inhomogeneities into simulations, 
but it may be an important future extension. 



16.2.2.3 Temporal Properties 



The rewiring taking place in the degenerating retina (see Chap. 3) and possibly other 
stages of visual processing are likely to include the loss of rapid signal processing 
and the creation of feedback loops. The effect of such changes will be that temporal 
properties of prosthetic vision will be much slower than in natural vision, as crudely 
represented in Fig. 16.6 by the "ghosting" of the visual percept of a maze. 
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Fig. 16.6 Tracing simulation 
shown in non-pixelized for- 
mat to facilitate visualizing 
the smearing of the maze and 
the circle indicating stylus 
contact when both stylus and 
head-worn camera are moved 




16.2.2.4 Dynamic Background Noise 

The occurrence of photopsias (see Chap. 5) in patients with retinal degenerations 
is expected to persist when these individuals receive a visual prosthesis. For this 
reason some simulations have examined the effects of the presence of randomly 
distributed and slowly decaying dots of light on visual task performance during 
simulations. Some of these spontaneous dots can be seen in the right panel of 
Fig. 16.4. 



16.2.2.5 Input Filtering/Windowing, Image Enhancement 

There is a wide range of other image manipulations that can be studied through 
simulations in the quest to improve prosthetic visual performance. Among the obvious 
examples are spatial and temporal filtering of the input image to adjust the image 
properties to those of prosthetic vision. Yet the opposite approach, pre-emphasis 
filters such as edge detectors may improve a prosthesis wearer's ability to detect 
obstacles or recognize objects by their outline. Some simulation studies (see Chap. 17) 
have dealt with the question whether a weighted input window for each unit cell in 
the simulation grid, c.q., in the prosthetic array, may be beneficial for image under- 
standing. Such studies have been few, and deserve further attention. 



16.3 Optotype Resolution and Reading 
16.3.1 Visual Acuity 

Cha et al.'s first experiments centered on measuring visual acuity using Tumbling 
E stimuli [3]. They provided their subjects with pixelized vision spanning up to 1.7° 
of visual field, using square grids varying in dot counts from 100 to 1,024. 
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Cha et al. examined decreases in dot count from the 1,024-dot condition both by 
holding the dot density intact, resulting in smaller grids, and by holding the grid 
size constant, resulting lower dot densities. 

Cha et al. found that visual acuity did not fall from that measured with the full 
1,024-dot grid (approximately 0.1 1 logMAR) when dot count decreased as long as 
dot density was maintained. Visual acuity fell to approximately 0.70 logMAR, 
however, when they reduced the dot count to 100 and reduced dot density accord- 
ingly. They concluded that the size of the dot grid, and therefore the span of the 
prosthetic vision, did not matter, and visual acuity was closely tied to the density of 
the dots provided. These investigators suggested that an array of 625-1,024 elec- 
trodes, corresponding to about 210-350 electrodes/deg 2 , could feasibly be placed in 
visual cortex to provide prosthetic vision with acuities as good as to 0.11-0.18 
logMAR. 

Between 2001 and 2003, Hayes et al., in a cooperative effort between the 
University of Southern California and Johns Hopkins University, examined visual 
acuity with prosthesis designs suitable for intraocular placement [21]. They created 
software implementations for 4x4, 6x10, and 16x16 dot grids, spanning 
7.3°x7.3°, 11.3°xl9.3°, and 19.3°xl9.3°, respectively. These measurements 
translate to each grid having respective densities of 0.30, 0.28, and 0.69 dots/deg 2 . 
These densities are much lower than those investigated by Cha et al. [3], but do 
reflect electrode densities possible for retinal prostheses with current technology. 

Subjects in this study performed with visual acuities of approximately 1.96 log- 
MAR with the 4 x 4 grid, 1 .82 logMAR with the 6 x 10 grid, and 1 .32 logMAR with 
the 16x 16 grid. Like in the Cha et al. study [3], the subjects used head motions to 
scan stimuli and achieve a better visual acuity than that possible with a comparable 
static image. Unlike the Cha et al. study [3], however, these subjects seemed to 
benefit from larger grid spans across the visual field, particularly noticeable when 
comparing the 4x4 and 6x 10 grids, which had nearly identical dot densities. This 
difference may be explained by the much sparser grids used in this study, and by 
the larger differences in grid size. Hayes et al. did confirm the importance of dot 
density, though, as the grid with the highest dot density offered the best visual acuity 
in this experiment. 

Cai et al., from Tsinghua University in Beijing, studied the effects of introducing 
irregularity in the presentation of simulated phosphenes, as could be expected of the 
percepts in actual prosthesis-wearers [2]. They used a grid with a density of approx- 
imately 0.86 dots/deg 2 , which offered visual acuities of about 1.36 logMAR. Using 
a scheme that assigned dot values without taking grid deformations into account, 
the irregularities induced a worsening of visual acuity by as much as 0.47 logMAR. 
Taking the deformations into account before down-sampling the original image, 
however, mitigated this acuity loss to only 0.22 logMAR. The authors therefore 
suggest that such a method of adapting down-sampling could help improve 
prosthesis-wearers' performance. 

Chen et al., of the Universities of New South Wales and Newcastle in Australia, 
have conducted the most detailed visual acuity research in relation to prosthetic vision 
simulations to date [6-10]. Unlike previous groups, Chen et al. have investigated 
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visual acuity using both grids with rectangular layouts and hexagonal layouts. 
Both grid layouts incorporated 100 dots in a square or approximately square grid. 
Dot densities remained at about 0.56 dots/deg 2 for hexagonal grids and rectangular 
grids that were shrunken to maintain density, and 0.51 dots/deg 2 for unaltered 
rectangular grids. 

In their first simulation study, published in 2004, Chen et al. examined visual 
acuity with hexagonal and unaltered rectangular grids by using Landolt rings [10]. 
Both pillbox and Gaussian input filters and dot profiles were implemented; all of 
these had circular apertures. For more discussion on these filters, see Chap. 17 and 
[18, 19]. The investigators also varied the effective size of the simulated phos- 
phenes. They found that the hexagonal layout seemed to provide an advantage over 
the rectangular layout, but not with statistical significance. For both filtering 
schemes, optimal acuities were reached at about 1.55 logMAR, at a sigma value of 
about 33% of the simulated phosphene separation for Gaussian filtering, and a 
kernel radius of about 50% of the dot separation for mean filtering. Visual acuities 
dropped to about 1.7 logMAR at non-optimal aperture sizes. These acuity measurements 
are reasonably consistent with the Hayes et al. and Cai et al studies, when considering 
grid densities. 

In a follow-up paper, Chen et al. took a closer look at grid configurations and 
densities [7]. They tested subjects on Landolt rings with the same grids as in their 
previous study, as well as the shrunken rectangular grid mentioned above. Simulated 
phosphenes were depicted as round dots with Gaussian profiles, and images were 
processed via mean filtering used in their first study. Similar to their previous 
results, measured acuities ranged from 1.55 logMAR to 1.7 logMAR; however, 
in this study, the authors were able to demonstrate a significant benefit (about 
0.5 logMAR) of hexagonal over non-shrunken rectangular grids for filter aperture 
sizes of 50 and 70% of dot spacing. The authors claimed to see some benefit the 
hexagonal grids over the rectangular grids shrunken to maintain density, but did not 
have enough subjects to make this difference significant. 

In a later paper [6], Chen et al. examined how subjects approached the simulated 
handicap of prosthetic vision. They first found that about 80% of subjects could 
improve their performance with prosthetic vision after practice. Their subjects 
improved by about 1.1% each session, which lasted an average of 33 min, up to 
about 15-20 sessions. 

As most subjects, during the course of testing with prosthetic vision, develop 
and utilize scanning to increase visual acuity, Chen et al. also investigated the types 
of scanning developed and their benefit over no scanning [8]. They found that fast 
(10°/s and greater), circular scanning motions helped subjects achieve acuities 
up to two times better than acuities inherently bestowed by the densities of the 
dot grids. This preference for circular scanning, however, may have been biased 
by testing only with Landolt rings, and no other stimuli. Horizontal and vertical 
scanning, as one of their subjects chose, may be just as or more beneficial in detecting 
non-circular targets. 

Finally, related to scanning techniques, Chen et al. also investigated the use 
of head movements while using prosthetic vision [9]. They first had subjects 
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perform the Landolt ring test with the simulator setup, but no actual simulation 
(i.e. simply performing the task with a video camera image). They found that head 
movements were minimal in these control tests, but subjects did adopt head motion 
when presented with simulated prosthetic vision. The speed of head motions seemed 
to increase by an average of 4°/s from the largest rings presented to the smallest 
(corresponding to 2.0-1.3 logMAR). The authors suggest that increased speed of 
head movements may provide greater access to high-frequency information than 
slower motions. 

Based on these simulation studies, the density of phosphenes perceived by 
prosthesis wearers should most directly affect the visual acuity they can achieve. 
At comparable density levels, greater spans of the phosphenes across the visual 
field should also provide some benefit [21]. Apart from the design of the devices 
themselves, practice [6], techniques of scanning [8, 9], and appropriately compen- 
sating for aberrations in perceived configurations should also affect the visual 
acuities achieved [2]. These conclusions must be taken with skepticism, however, 
as simulations do not necessarily reflect percepts elicited by real prostheses. 
Even simple assumptions, e.g., that phosphene resolution improves with increased 
electrode density, deserve serious scrutiny: More densely packed electrodes would 
be required for higher-resolution percepts, yet there is no guarantee that electrodes 
placed so close together will create separate and distinct phosphenes. Moreover, 
these studies did not incorporate any thorough measures to restrict or monitor eye 
movements, and thus may have overestimated the resolution of prostheses with 
external image capture [11]. 



16.3.2 Reading 

Cha et al.'s study of reading with simulated prosthetic vision [5] used the same 
setup as their visual acuity study [3]. Subjects read text that was either scrolled as 
they read, so no scanning was necessary, or used head or eye movements to scan a 
page of text. Without any complications of scanning, the subjects could read text, 
where letters had an optimal size of approximately 0.4° of visual angle, at rates of 
200 words/min with grids containing 625 or 1,024 dots. Reading rates dropped 
significantly with lower dot counts, but interestingly, this did not depend on 
whether grid size or density was varied. Thus, unlike in visual acuity, the authors 
found that dot count is the primary factor in determining reading speed. 

This was confirmed by tests using head scanning to read a page of text; reduc- 
tions in dot count had the same effect for both reducing dot density and reducing 
grid size. Tests with head and eye scanning did additionally show, however, that 
the requirement of scanning significantly slows subjects' reading speeds. With 
head scanning, reading rates dropped to 120 words/min for grids containing 625 
or 1,024 dots. Scanning text with eye movements proved to be more difficult, 
with reading rates of only 55 words/min, but the exact reason for this seems to 
be unclear. 
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Dagnelie et al., of Johns Hopkins University, followed Cha et al.'s study with a 
publication on reading with prosthetic vision in 2001 [14]. Like Cha et al., they 
found that reading speed did increase with increasing numbers of dots in the grids. 
Dagnelie et al. tested other parameters, as well, including the effects of dot dropout, 
high versus low contrast, number of gray levels, dot size and spacing, and font size. 
The authors found that even with 30% dot dropout, reading rates of 50 words/min 
were possible with grids containing 625 dots. Increasing dropout, reducing con- 
trast, or altering character perception to less than 2 cycles/character width (i.e., 
4 dots/char width) all generally slowed reading rates. The number of gray levels, 
however, did not appear to have a significant effect. Although the free-viewing 
reading rates published in this study are markedly lower than those reported by Cha 
et al. [5], the results are comparable when taking into account that Dagnelie et al. 
did not perform any tests with less than 30% dropout. 

Sommerhalder et al., from the Geneva University Hospitals, Switzerland, in 2003 
conducted reading tests [26] with a different approach than the previous two groups. 
Unlike Cha et al. and Dagnelie et al., Sommerhalder et al. projected a stabilized 
image onto the retina and denied subjects the option of scanning with eye move- 
ments throughout their experiments. Their reading tasks also differed, in that sub- 
jects were only required to read one word at a time, and subjects could not change 
their perspectives of the word by scanning. Under these conditions, the authors 
concluded that grids of at least 300 dots located over the fovea would be necessary 
for reading accuracies greater than 90%. For eccentricities of 10° or more, grids of 
greater size and/or resolution become increasingly necessary to maintain high read- 
ing accuracy. For example, they found that a grid located at 20° eccentricity and 
spanning 20° x 7° required about 875 dots to achieve 63% reading accuracy. 
Sommerhalder et al. found that problems at high eccentricities relate to a "crowding 
effect," in which interference among stimuli reduces perception below the other- 
wise expected visual acuity. The authors suggest that training with eccentric reading 
can improve performance, through suppression of this "crowding effect" and/or 
reduction of reflexive eye movements. 

Sommerhalder et al. expanded their study in 2004 to incorporate full-page reading 
[27], more akin to the experiments already conducted by Cha et al. and Dagnelie 
et al. For these experiments, the authors continued to enforce a gaze-stabilized 
view, and chose to model a optoelectronic retinal prosthesis, with which subjects 
could use eye movements to change the image presented by the array. When they 
stabilized the image of the grid, containing 572 dots spanning 10° x 7° of visual 
field, over the fovea, they measured reading performance similar to that found by 
Cha et al. when eye-tracking was employed, about 65 words/min and nearly perfect 
reading accuracy. One of their three subjects was able to improve to a reading rate 
of 122 words/min, but neither of the other two subjects showed significant improve- 
ment with practice. 

In the second phase of their 2004 study, Sommerhalder et al. stabilized the 
image of the grid at a visual eccentricity of 15° below the center of vision. At first, 
reading rates dropped to about 3 words/min, with reading accuracies of 85% for one 
subject and 13% for the other two. After approximately 2 months of practice, 
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55-68 sessions per subject, reading accuracies plateaued at an average of 94%. 
At this time, reading rates were still improving, and the final measured rates aver- 
aged 23 words/min. During these experiments, the authors also measured reading 
comprehension, and found that useful reading ability most likely requires at least 
85% reading accuracy. The authors thus concluded that, in agreement with previ- 
ous studies, the prosthetic generation of 600 discrete percepts should be sufficient 
to allow effective reading. An array of the same size placed as far out as 15° eccen- 
tricity would also be usable, but would require significantly more instruction and 
practice. 

Kelley et al., of the Johns Hopkins University group, presented a study on reading 
with gaze-locked vision in 2004 [23]. They provided subjects with grids of lOx 10 
or 25 x 25 dots to view segments of text, which the subjects scrolled by controlling 
a mouse. The authors varied dot size, font size, contrast, and the application of 
image stabilization. The subjects in this study maintained reading accuracies of 
95% or more for nearly all forms of free- view reading; only when characters 
spanned as few as 4.5 dots, accuracy fell to 70-80%. In gaze-locked trials, accuracy 
still remained above 90% for most conditions, and fell to about 60% for characters 
spanning 4.5 dots. Reduction of character resolution caused the greatest drop in 
reading speed, about 60-80%, followed by stabilization, reducing speed by 
50-75%, and contrast reduction lowered speeds by 15-25%. Dot size did not seem 
to have a significant effect. The authors also point out that practice significantly 
improved performance, particularly for trials using stabilization or low contrast, as 
demonstrated in Fig. 16.7. This study points out the need for sufficient character 
resolution to have successful reading with simulated prosthetic vision; more impor- 
tantly, it demonstrates that adequate training will allow reading under gaze-locked 
conditions, reducing concerns expressed by many that performance of complex 
tasks would be impossible with external image acquisition, unless eye movement 
feedback is built into the system. 

Dagnelie et al. performed a another study on reading with prosthetic vision in 
2006 [12]. Similar to their 2001 study [14], the authors tested the effects a wide 
variety of parameters on paragraph reading controlled by mouse scrolling, with no 
restrictions on viewing. These parameters included dot size, spacing and count, 
dropout, number of gray levels, contrast level, and text size. The authors found 
significant effects of each of these parameters, and determined that reading with 
90% accuracy was possible provided that characters were at least three dots wide 
and dropout levels did not surpass 50%. Reading speed dropped below 20 words/ 
min whenever accuracy fell below 90% or, at low contrast, the presented grid was 
smaller than the width of two characters. The authors suggest that an array containing 
256 electrodes, with 30% dropout, should be sufficient to allow accurate paragraph 
reading, with a maximum reading rate of about 30 words/min. 

According to these studies, reading with prosthetic vision should certainly be 
possible. Reading accuracy appears to depend strongly on the resolution provided 
to the visual system, requiring at least three distinct phosphenes across for each 
character [12]. Provided useful reading accuracy (i.e. at least 85% accuracy [27]), 
reading rates seem to depend most upon the number of distinct phosphenes a 
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Fig. 16.7 Reading accuracies (top) and speeds (bottom) for two stabilization (free-locked) and 
two contrast (low-high) conditions [23]. Trials were presented in six sets (1-6), each composed of 
three consecutive blocks of 16 trials (A-C). Each set took between one and three 1-h sessions to 
complete. Error bars denote the between-subject standard deviation among five subjects. The "low 
vision" points represent the performance of the one subject with severely reduced visual acuity 
and contrast sensitivity. Notice the effect of practice in on gaze-locked performance 

prosthesis-wearer can perceive [5]. Without any dropout, arrays producing about 
600 distinct percepts should be enough to allow read rates of about 50-70 words/ 



16.4 Face and Object Recognition 



Dagnelie et al. published the first study involving a recognition task in 2001 [14]. 
In their experiment, subjects were asked to identify pixelized faces 12° wide among 
four options. The number of dots in the grid simulation, percentage of dot dropout, 
and the number of gray levels strongly affected subjects' ability to recognize these 
faces. Instances where grid parameters reduced the sampling frequency below 8 
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cycles per face width, grid size fell below 256 dots and 7 deg 2 , dropout exceeded 
50%, or when fewer than four gray levels were used, caused face recognition abili- 
ties to fall to chance. Recognition accuracy of 80% or greater was consistently 
possible only in 98% contrast conditions, specifically with a grid size of 625 dots 
spanning ll°xll°, or a grid of 256 dots about 70' wide each. 

Thompson et al., of the same Johns Hopkins University group, with collabora- 
tors from the University of Southern California, published a very similar study to 
the one above in 2003 [30]. In their 2001 experiment [14], however, they used a dot 
size of 23' of visual field as a baseline parameter, whereas the base dot size was 
increased to 31.5' in their 2003 experiment. Among other factors, this increased the 
basic grid size from 7°x7° to 9.6°x9.6°. While no parameter combination gener- 
ated an average accuracy of 80% or more when contrast was set to 12.5%, several 
parameter conditions with 99% contrast did allow facial recognition with 80% or 
more accuracy. Based on their results, it appeared that a dot density around 1 dots/deg 
with 4.5-arcmin dot spacing was optimal among their parameter combinations. 
Increasing the grid size from 256 dots to 625 and 1,024 dots also consistently 
improved recognition accuracy. 

Hayes et al. [21] also reported on object recognition using prosthetic vision 
simulations. Using grids with 4x4, 6x 10, or 16x 16 dots, with various levels of 
contrast and dynamic noise, subjects were asked to visually describe and, if possi- 
ble, identify common objects without touching them. Contrast and noise did not 
appear to have significant effects, but grid size did. The 4x4 and 6x 10 grids had 
the same dot size and spacing, but the 6x 10 grid provided a significant advantage 
over the 4x4 grid. This would suggest that grid size, measured by dot count or 
visual span, is important for recognition tasks. The 16 x 16 grid was significantly 
more useful for object recognition than the 6x 10 grid, but this could also be an 
effect of increased dot density. 

Dagnelie et al. published a study on visual discrimination of white target squares 
on a black background ("modified checkerboard") in 2006 [15]. While this study 
did not ask subjects to recognize specific features of these targets, it did evaluate 
the subjects' abilities to discern and count these targets when gaze-locking was 
employed. For most of the subjects, the time to count all the targets on a modified 
checkerboard did not change with the number of targets, as a result of their counting 
strategies, but the addition of gaze-locking did significantly increase counting time, 
particularly before practice. Srivastava et al., of the Illinois Institute of Technology 
in cooperation with Dagnelie et al., published similar experiments with this counting 
task in 2009 by simulating a cortical prosthesis [29]. In these experiments, gaze- 
locking was enforced consistently and levels of dropout varied so that 325-650 dots 
were used. Search times were comparable with the 2006 study [15], after practice, 
and levels of dropout did not seem to affect the performance of most subjects. 

Zhao et al., of Shanghai Jiao Tong and Peking Universities in China, reported 
results of testing subjects with object and scene recognition tasks in 2008 [34] and 
2010 [33]. Subjects could freely view 4.5° x4. 5° grids of either square or circular 
dots with various dot densities and two different methods of image processing: 
binary (black-white) output through contrast enhancement, and edge detection. 
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The authors set their threshold of recognition at 60% accuracy. They found that 
for common objects, subjects passed this threshold between grid resolutions of 
16 x 16 and 24x24 dots, and for scenes, subjects reached this threshold around a 
grid resolution of 48x48 dots. When using a slightly lower grid resolution, the 
authors observed that simple binary image processing was more helpful than use of 
edge detection. At higher resolutions, however, edge detection seemed to be more 
beneficial. 

Recognition tasks, in general, appear to be sensitive to the number dots con- 
tained within a grid [21, 33, 34]. Particularly for faces, high image quality of near- 
100% contrast, at least four gray levels, and at least 256 dots appears to be required 
for 80% or greater recognition accuracy [14]. This is not surprising, since the 
coarse traits of faces resemble each other, and successful discrimination is based on 
finer traits and shading. 



16.5 Visually Guided Behavior 
16.5.1 Hand-Eye Coordination 

Along with their early visual acuity and facial recognition tasks. Dagnelie and 
colleagues conducted experiments of hand-eye coordination using both virtual 
reality and live video input [14]. In their virtual reality experiment, four subjects 
viewed a room with a table and chairs through pixelized vision with less than 20% 
contrast. Their task was to pick up objects off the floor and place them on the table, 
releasing them only when they rested on the surface. Out of 12 total attempts to 
transfer objects in this virtual scene, subjects were successful six times. In the live 
video experiment, with about 90% contrast and a 250 ms delay, subjects were able 
to transfer objects with the assistance of tactile feedback. Technological limitation 
prevented systematic studies, but these experiments did serve as an example for 
how prosthetic vision could be used for coordination tasks. 

The same group later expanded their study of hand-eye coordination with simu- 
lated prosthetic vision, reported in Hayes et al., 2003 [21]. In this study, subjects 
were asked to perform two tasks. The first asked them to pour ten pieces of candy 
from one cup to another, without touching the second. Some subjects were able to 
do this successfully in the hardest condition, using a grid of only 4x4 dots. Only 
one subject required a grid of 16 x 16 dots. The authors concluded that, on average, 
a 6x 10 grid of dots would be sufficient for a simple hand-eye coordination task. 

The second task asked of these subjects was to cut along the outside of a black 
square outline on a white piece of paper. Times to completion and errors both fell 
with increasing grid size, where satisfactory performance was only achieved with a 
16x16 grid. Hayes et al. reasoned that this task, unlike the first, requires constant 
reevaluation to acquire the position of the scissors relative to the border. The 
authors thus concluded that for more complex tasks the 6x10 grid would be insuf- 
ficient, and larger and/or denser grids would be required for acceptable performance. 
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Dagnelie et al. and Srivastava et al. continued their studies with modified check- 
erboards, respectively in 2006 [15] and 2009 [29], by asking subjects to cover these 
targets with black checkers. Once a subject had correctly covered a target, the target 
would no longer be visible through the simulation grid. Dagnelie et al. found that, 
using a 6x 10 grid, subjects could successfully learn how to perform this task with 
as little as 0.85% error, even in gaze-locked conditions. 

Srivastava et al. utilized gaze-locking throughout their experiment (see Chap. 18 
and [29]), and varied dot counts between 325 and 650 dots. Similar to the results of 
the counting task, practice seemed to substantially reduce any effects of dropout on 
time or error. After practice, these subjects were able to complete the task without 
any errors through the simulated cortical prosthesis. 

As suggested by the Hayes et al. study [21], hand-eye coordination tasks seem 
to benefit from increases in grid size. It is unclear, however, whether this benefit 
is derived from a greater dot count, the increased visual span of the grid, or both. 
As seen with Dagnelie et al. [15] and Srivastava et al., gaze-locking and dropout in 
simulations do seem to mandate practice if normal performance is desired, but do 
not strongly hinder performance after the initial learning period. 



16.5.2 Wayfinding 

Cha et al. [4] provided subjects with simulated prosthetic vision similar to that in 
their visual acuity and reading experiments. The authors varied dot density, dot 
count, overall grid size (up to 1.7° x 1.7°), and the visual angle captured by the 
camera and projected onto this simulated prosthesis. They asked subjects to navi- 
gate through a maze with white walls, floor, and ceiling and black obstacles. The 
capture angle was found to be critical for this task: performance increased with this 
angle so long as individual stimuli did not become too small; performance declined 
once the capture angle was expanded past 1 8 times the angular subtense of the grid. 
At the optimal viewing angle, performance correlated well with the number of dots, 
almost regardless of dot density. The authors concluded that a cortical prosthesis 
with 25 x 25 or 32 x 32 electrodes, perceptually spanning 1.7° and incorporating 30° 
of a camera's view, could be used effectively for high-contrast obstacle avoidance 
and wayfinding in a familiar environment. 

Dagnelie et al. published a report in 2007 of a similar pair of experiments on 
wayfinding [13]. In the first experiment, subjects used 4x4, 6x10, and 16x16 grids, 
respectively spanning ll°xll°, 16°x27°, and 27°x27°. The camera's viewing 
angle was fixed at 37°. The authors found that, with increasing dot count, the sub- 
jects' wayfinding performance improved. For experienced subjects, the 6x 10 grid 
was sufficient for this task. In their second experiment, subjects used the 6x 10 grid 
to navigate through a virtual environment and were presented additional parameters 
of dynamic noise and dot dropout. The authors observed that noise did not have a 
significant effect, and dropout of 30% led to a slight decrease in performance. These 
findings match well with those of Cha et al. [4], particularly as the only differences 
between the 4x4 and 6x 10 grids were size and dot count, and not density. 
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In 2008, Boyle et al., of Queensland University of Technology in Australia, con- 
ducted experiments where subjects were asked to judge the quality of pixelized 
images [1]. No viewing constraints were used, and no actual wayfmding was per- 
formed in this study. Subjects, instead, viewed an original image and then judged 
which among a set of binary, 25 x 25 dot versions of that image they would find most 
useful for navigation. The authors applied various algorithms to highlight important 
or salient stimuli in an image, and investigated various approaches to zooming the 
image in on important features. The authors found that subjects did not favor any 
kind of feature-based processing when no zoom was applied. In the second phase of 
their study, however, subjects appeared to prefer a zooming method that trimmed 
away unimportant pixels based on a saliency map generated by a program from the 
University of Southern California. The final image was still 25x25 dots, but con- 
tained information from a smaller portion of the original image. The authors suggest 
that such saliency detection and zoom could be used for actual prosthetic devices. 

Wang et al. in our laboratory published experiments on wayfmding in virtual 
environments later in 2008 [32]. Subjects viewed a virtual environment through a 
stabilized image of 6x 10 dots spanning 16.2°x27°. The authors investigated factors 
affecting time to traverse the virtual environment, including contrast, background 
noise, and dot dropout. Of these, only dot dropout had significant effect on perfor- 
mance. When dropout was set to 30%, completion time increased by 40%. This 
accents the importance of the number of dots available to subjects for wayfinding, 
as well as the importance of array integrity in visual prosthesis implantees. 

Srivastava et al., in the same publication as their target counting and checker 
placing tasks [29], had subjects navigate similar virtual environments as those of 
Wang et al. and Dagnelie et al., using a cortical rather than a retinal grid layout. 
Unlike with Wang et al. and Dagnelie et al., however, these subjects did suffer a 
significant deterioration in performance with dot dropout. This may be a result of 
the large visual angle subtended by the cortical grid, causing a random reduction 
from 650 to 325 dots (at 50% dropout) to leave a sparser set than what was available 
in retinal simulations. 

Jointly, these studies suggest that, so long as an optimal viewing angle is 
obtained [4], wayfinding performance is primarily dependent upon the number of 
dots presented in a grid simulation [4, 13]. Based on Dagnelie et al.'s [13] and 
Wang et al.'s [32] studies, current 6x10 electrode arrays placed in the retina should 
be sufficient for basic wayfinding, but performance will deteriorate in cases of 
significant electrode dropout. 



16.6 Visual Tracking 

Hallum et al., of the Universities of New South Wales and Newcastle, Australia, 
examined the abilities of subjects to track a target using grids generated with three 
different aperture weighting schemes [20]. Each of these grids contained 23 dots in 
a hexagonal layout, spanning about 7.4° x 5.4°, on a screen spanning 16.7° x 16.7°. 
A target sized 36 arcmin 2 moved across the screen, generally in an S-shaped 
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pattern, and subjects were asked to keep the grid centered over the target by moving 
a joystick. The first filtering scheme the authors employed was analogous to the 
setup of Cha et al. [3-5], where the stimulus was only sampled at the 23 locations 
corresponding to dot positions. The second scheme regionally averaged the under- 
lying stimulus to determine dot characteristics. The third scheme employed a 
Gaussian sensitivity profile with a = 34.4'. Subjects could freely view the screen 
binocularly. The authors found that, after practice, the use of a Gaussian profile 
permits more accurate fixation on, saccades to, and pursuit following the targets. 
The authors advocate use of this filtering technique to actual prostheses to improve 
performance in visual tasks, but they do not discuss the possibility that the unre- 
stricted gaze and the choice of stimuli may have skewed their findings. 

Wang et al. published a study with a similar experimental setup in 2008 [31]. 
Subjects viewed, monocularly, an 0.94° target moving across a screen, either with 
natural vision or through a 10 x 10 cell grid. The subjects were asked to follow the 
target with their eyes, monitored by an infrared eye tracker. When simulating pros- 
thetic vision, the rendered grid would move with the subject's eye movements, 
corresponding to an array placed over the fovea, superior retina, or nasal retina. The 
authors found that, compared to using natural vision, subjects took about 65% lon- 
ger to detect sudden target movements in simulated prosthetic vision. This is com- 
parable to the 20% increase in reaction time over natural vision reported by Hallum 
et al. [20], when considering differences in viewing conditions between these two 
studies. The authors also found that horizontal eye movements were more unstable 
in simulated prosthetic vision, particularly for stabilized stimulus placement on the 
nasal retina. Simulating superior retinal placement offered more stability than nasal 
placement, but not as much as foveal placement. The authors conclude that, 
although initiation is slower and movement is less stable, pursuit eye movements 
should be possible with optoelectronic retinal prostheses and that, among periph- 
eral retinal locations for a prosthesis, superior placement may be more beneficial 
than inferior placement. 



16.7 Computational Simulations 

Pezaris et al., of Harvard Medical School, are pursuing a visual prosthesis within 
the lateral geniculate nucleus (LGN) of the thalamus [25]. In 2009, these authors 
published results of a study they conducted to evaluate four different basic prosthe- 
sis designs with various values of electrode spacing. Unlike all other simulation 
studies mentioned in this chapter, however, Pezaris et al. used a purely computa- 
tional approach to determine the likely benefit of each design's implementation. 

The authors specifically investigated prosthesis designs where electrode tips 
are placed in a 3D grid throughout the LGN or along a 2D slice through LGN, 
and where activation is brought through two different forms of deep brain stimu- 
lation (DBS) electrodes. According to their analyses, the 3D grid would be the 
most useful for creating many phosphenes throughout the visual field. Among the 
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possibilities for laying out a 2D grid in LGN, with much less of the visual field 
being stimulated, placement along a sagittal slice midway through the LGN would 
seem to be best. DBS electrodes that can contain as many as 60 microwires could 
create stimulation similar to the 2D plane. Traditional DBS electrodes, however, 
would only offer three to four large conductive cuffs on their shafts, which would 
provide stimulation more akin to solid curves or arcs. While the implementation of 
an LGN prosthesis through a traditional DBS electrode may be technically and 
surgically simpler, it would provide little useful stimulation. On the other hand, a 
full 3D grid, which would provide selective stimulation for the entire visual field, 
would be much more challenging. Future studies may shed more light on exactly 
how much benefit each design could provide. 



16.8 Conclusion 

Many of the designs found by these simulation studies to offer desirable performance 
in common visual tasks exceed current technological realities. For example, numer- 
ous reading studies concluded that grids containing about 600 dots are required for 
reading speeds of 50-70 words/min. Some studies, however, purposely used grid 
configurations that are currently in use by visual prostheses. These studies, limiting 
grid sizes to about 4x4 or 6x 10 dots, found that even simple prosthesis designs 
can be used for modest performance on everyday visual tasks. While not as appli- 
cable to guiding performance expectations for current devices, simulations that 
represent more complex designs do guide developers in expanding this technology 
and provide specific goals for prosthesis structure and function. 
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Chapter 17 

Image Analysis, Information Theory 

and Prosthetic Vision 

Luke E. Hallum and Nigel H. Lovell 



Abstract Recent years have seen markedly improved clinical outcomes in cochlear 
implantees. This improvement is largely attributed to improvements in speech pro- 
cessing algorithms. In light of these improvements, researchers are prompted to ask, 
"Could image analysis improve clinical outcomes in retinal implantees?" We discuss 
our approach to image analysis, microelectronic retinal prostheses, and the percep- 
tion of low-resolution images, which we believe can be used to help constrain the 
design of an implant. We hope that our approach, and developments thereof, will 
ultimately contribute to improved clinical outcomes in retinal implantees. 



Abbreviation 

APRL Artificial preferred retinal locus 



17.1 Introduction 

It makes intuitive sense that the cochlear implant involves a speech processor. This 
processor is typically implemented in programmable microelectronics that the subject 
wears behind the ear; it lies between the device microphone and the electrode array 
implanted in the subject's cochlea. The processor analyzes incoming acoustic signals, 
deriving parameters that determine the current waveforms injected at each electrode. 
Recent years have seen markedly improved clinical outcomes in cochlear implantees. 
This improvement is largely attributed to improvements in speech processing algo- 
rithms [9]. In light of these improvements, researchers are prompted to ask, "Could 
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image analysis - analogous to the cochlear implant's speech processing - improve 
clinical outcomes in retinal implantees?" This chapter is concerned with image analysis 
and the perception of low-resolution images. 

In this chapter we review two studies of ours [6, 8]. In the first study prosthetic 
vision was simulated via low-resolution images that we refer to as "phosphene 
images." Normally sighted subjects were required to track a moving target. The 
phosphene images were rendered using three image analysis schemes, and we exam- 
ined the effects of each scheme on subjects' tracking performance. The second study 
was numerical. We used information theory to quantify the amount of information 
contained in the phosphene images, and how different image analysis schemes affect 
this information. This numerical approach ultimately served as a reasonably good 
model of subjects' tracking performance in [8]. Further, we used this numerical 
approach to suggest how image analysis schemes could be further improved. 

The two above-mentioned studies help constrain the design of a retinal prosthesis. 
They directly address the topic of image analysis and its integration with a retinal 
implant. They effectively ask, "How should an implant process incoming images?" 
This contribution is taken up in the Sect. 17.5. There, we also discuss generalizing the 
approach to visual tasks beyond fixation, saccade, and pursuit, for example, visual 
acuity tasks and reading. We begin this chapter by situating image analysis, function- 
ally speaking, with respect to the prosthetic device as a whole, and by describing the 
experimental framework within which low-resolution perception is investigated. 

17.2 Situating Image Analysis 

The major components of a retinal prosthesis, as it is envisioned and prototyped by 
a number of groups [4], are the camera, the image analyzer, the communication link 
and the implant, as discussed elsewhere in this volume. Physically, the image ana- 
lyzer is external to the body and, functionally, lies between the acquisition of high- 
resolution images of the world ("scenes") and the generation of signals for 
transmission to the implanted component of the device (via the communication link). 
The camera, worn on spectacles by the subject, captures high-resolution spatiotem- 
poral images of the real world. The image analyzer is implemented in program- 
mable microelectronics, a digital signal processor, worn on the body. It processes 
data captured by the camera, effectively converting scenes to an array of numbers 
which determine the current waveforms at each electrode. These numbers are deliv- 
ered to the implant by the communication link. As with the cochlear implant, this 
link ideally comprises radio-frequency signals transmitted transcutaneously; as 
opposed to a percutaneous link, a radio-frequency link poses a lesser risk of infec- 
tion and allows the external unit to be physically uncoupled from the body. The 
implanted analog electronics decode the incoming radio-frequency signals and 
drive an array of electrodes. These electrodes are embedded in a flexible substrate 
that is affixed to the inner layer of the retina. Recent clinical trials involve arrays of 
60 electrodes (Argus™ II, http://clinicaltrials.gov/ct2/show/NCT00407602). 

Here we will step the reader through the processing stream that generated the 
phosphene image in Fig. 17.1. First, the high-resolution image (left panel) was filtered. 
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Fig. 17.1 A basic simulation of prosthetic vision. The high-resolution image (left) is filtered and 
subsampled. The samples are used to modulate the sizes of luminous phosphenes which, together, 
comprise the phosphene image (right). Image source: http://pics.psych.stir.ac.uk/ 



This filtering anti-aliases the resulting phosphene image, and may be used to 
emphasize salient features of the high-resolution image. Second, the filtered image 
was sampled at 400 locations, each pertaining to a phosphene. Each of these samples 
was then quantized to one of 16 levels, and used to modulate the size of its corre- 
sponding phosphene. Quantization makes for more accurate simulation since, as 
with the cochlear implant, the microelectronic retinal prosthesis is likely to elicit 
only a small number of different percepts at each location in the visual field. 



17.3 The Experimental Framework 



The right-hand panel in Fig. 17.1 shows a phosphene image of a face. This image 
depicts the sort of vision that the microelectronic retinal prosthesis aims to provide 
the otherwise profoundly blind implantee, that is, a relatively small number of dis- 
crete, luminous blobs. Phosphene images like this one, and also phosphene images 
that vary over time, may be used in psychophysical experiments and presented to 
normally seeing observers. These experiments draw on well established, psy- 
chophysical methods in vision research wherein perception is inferred through the 
measurement of subjects' behavior. For example, the reading speed of subjects 
presented low-resolution, phosphenized text could be measured. In this way, 
experimenters may use simulation data to predict clinical outcomes in actual 
implantees. This approach, which we call "visual modeling," is analogous to 
"acoustic modeling" of cochlear implants wherein colored noise bands, modulated 
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by speech, or in recent cases by music, are played to normally hearing listeners. 
These listeners form a more readily accessible cohort than one comprising actual 
cochlear implantees. The behavior of these normally hearing listeners is then used 
to investigate improved signal processing strategies and electrode array designs. 

Visual modeling is an experimental framework that allows us to test predictions 
concerning image analysis, microelectronic retinal prostheses, and the perception 
of low-resolution images (for discussion, see [7]). It has been used extensively in 
the field since the early work of Cha [2], who argued that "three parameters are 
important in determining the quality of a pixelized image: the number of pixels, 
their density, and their range of intensities." By contrast, we are using the approach 
to test image analysis schemes. 



17.4 Tracking a Low-Resolution Target 

Several years ago we wondered whether image analysis could improve the perfor- 
mance of the phosphene image observer [8]. To this end, we conducted a visual 
modeling experiment that compared the effects of three different image analysis 
schemes on subjects' performance. The task involved the fixation of, saccading to, 
and the pursuit of a small, high-contrast target. The first image analysis scheme that 
we investigated, referred to as scheme QO, was trivial: images were not prepro- 
cessed but simply down-sampled. This scheme allowed for the comparison of 
Kichul Cha's results [2] with our results. The second scheme (Ql) blurred images 
using a uniform-intensity filter kernel, that is, the spatial equivalent of a boxcar 
filter. The spatial width of the kernel was identical to the spatial separation of pho- 
sphenes. This scheme allowed for the comparison of our results and the widespread 
approach to visual modeling which uses uniform-intensity kernels [5, 10]. The third 
scheme (Ql) involved pre-filtering with a Gaussian kernel. The standard deviation 
of the kernel was equal to one-third of the separation of phosphenes. We were 
interested in this kernel due to the nature of a Gaussian: it is dually compact in the 
spatial and Fourier domains, and is often used to model components in the early 
visual system of mammals. The standard deviation of this Gaussian, however, was 
chosen without quantitative reasoning. We hypothesized that scheme Ql would 
afford subjects improved performance as compared to Ql. 

For complete details as to the experiment the reader is referred to the original 
publication [7]. Here we summarize the details. A computer monitor was viewed 
from a fixed distance by 20 subjects each trained for 3h. A hexagonal array of 23 
phosphenes was freely viewed. Phosphenes were separated by approximately 1°, 
and therefore excited retinal loci separated by approximately 300 urn. Phosphenes 
were size-modulated (see Fig. 17.1), and of fixed intensity (white). The phosphene 
array was moveable via a joystick; subjects were required to track a moving target 
(a small, white square on a black background) using the central phosphene of the 
array. The target initially appeared in the center of the monitor (for fixation), and 
after a short, random interval it jumped (eliciting a visuomanual saccade in the 
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subject) and then described a randomly generated, S-shaped course into the monitor's 
periphery (eliciting pursuit) at an average velocity of approximately 2°/s. Trials 
were counterbalanced so as to assess tracking for each of QO, Ql, and Q2. Note that 
the setup, whilst good for examining the issues at hand, measured visuomanual 
behavior. Ultimately, prosthetic visual fixation, saccade, and smooth pursuit will likely 
involve head movements (of a head-mounted camera) and a stabilized retinal 
image. Thus we take several of the statistics of the measured behaviors as a model 
for what will ultimately involve head motion and stabilized retinal images. 

Our data showed that, indeed, image analysis had a significant effect on the 
tracking performance of subjects. After practice, schemes Ql and Q2 made for 
superior performance in all tasks (fixation, saccade, and pursuit) as compared to 
QO. Scheme Ql made for improved fixation accuracy of 35.8% (8.3 min of visual 
arc) as compared to scheme Ql (as measured by mean deviation from the target), 
and for improved pursuit accuracy of 6.8% (3.3 min of visual arc). Schemes Ql and 
Q2 made for comparable saccade accuracy. These results suggest that image analysis, 
when functionally integrated with a prosthetic device, can be programmed so as to 
result in improved visual outcomes in implantees. Furthermore, the results advocate 
a scheme of Gaussian kernels for fixation- and pursuit-related tasks. 

The scanning data from the above-described experiment, that is, the way that 
subjects moved the phosphene array relative to the moving target, were also of interest. 
These data made us think about preferred retinal loci, and whether implantees would 
use some phosphenes in their visual field in preference to others. Hence, we coined 
the term "artificial preferred retinal locus" (APRL). 1 In the case of scheme QO, 
which afforded subjects poor tracking accuracy, subjects adopted nystagmus-like 
scanning behaviors, that is, vigorous and wide-ranging scanning. This was appar- 
ently an attempt to effectively increase the spatial sampling rate of the array and in 
doing so render the phosphene image more informative as to the moving target's 
location. In this case, the associated APRL may be modeled as a bivariate function, 
centered on the array's central phosphene. That function is uniform in intensity, 
covering an area that encompasses most of the 23-phosphene array. 

The scanning associated with scheme Ql was similar to that of Q2. For those 
schemes, subjects adopted scanning that had dynamics approximately equal to the 
target's motion. In terms of an APRL, the bivariate function was centered on the 
central phosphene of the array, and was normal with a standard deviation equal to 
approximately 0.475°. Since tracking for Q2 was more accurate than tracking for 
Ql, the APRL associated with scheme Q2 was relatively narrow. See Fig. 17.2. We 
believe that scanning and modeling the APRL have important implications for 
developing a model of the phosphene observer, which we discuss further below. 

Subsequently to the above-described experiment, we wondered, "Is scheme Q2 
somehow optimal?" To address this question, we drew upon techniques in com- 
munication theory [6]. Specifically, we used the mutual-information function to 



1 For example, see the way in which sufferers of scotoma develop new preferred retinal loci in the 
vicinity of the fovea [11]. 
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Fig. 17.2 Raw and analyzed scanning data from a single trial [8]. (a) Subjects were required to 
track a small, moving target with a phosphene array. The target's motion is shown by the thin, 
solid line. The target was initially stationary at the center of the monitor. The target then stepped 
(down and right by approximately 2°) before following an S-shaped curve to the monitor periphery. 
The thick, dotted line shows the subject's tracking. Specifically, the thick, dotted line shows the 
location of the center of the phosphene array. Each dot shows the position of the array center at 
successive points in time, (b) The scanning signal (target position minus tracking position) from 
the trial in (a). The average of this signal across trials and subjects is well modeled by a bivariate 
normal, indicated by the histograms for this trial 



measure the amount of information contained in phosphene images, and how that 
information differed with different image analysis schemes. We used a numerical 
setup. We presented targets to the phosphene image in a way that was consistent 
with scanning behaviors, that is, the APRLs, found in [8]. These mutual-information 
measurements were then reconciled with the tracking performance. We found 
that, to an extent, the mutual-information function was a good model of tracking 
performance. 

As a quick aside, the following paragraphs canvas the mutual-information func- 
tion [1]. The mutual-information function is typically applied to communication 
channels, like the one shown in Fig. 17.3. There, a time-varying signal, x(t), is input 
to a noisy channel, Q, and output in modified form as y(t). The mutual-information 
function can be used to measure the information that y(t) carries about x(t). In other 
words, after receiving the signal y(t), what is consequently known about the signal 
x(t)l Mutual information is typically used to assess the nature of the channel, Q. 

The mutual-information function is written as follows: 



I(p(x(t)); Q) = H(y(t)) - H(y(t) | x(t)). 
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Fig. 17.3 A simple communication channel. The input signal, x(t), to the noisy channel, Q, results 
in the output signal, y(t) 



The symbols / and H may be read as "information" and "uncertainty" respec- 
tively. Note that information is a function of the probability density that describes 
the input, p(x(t)). Information is also a function of Q, that is, the nature of the channel. 
Information is equal to the receiver's reduction in uncertainty regarding the channel input 
after having received the channel output (the term y(t)\x(t) may be read as "the 
input after having received the output"). 

To illustrate the use of the mutual-information function, and its interpretation, 
consider the following example. A discrete random series, X, may take values a, b, c 
or d. All four values occur independently and with equal probability (p = 0.25). The 
series X generates an output series, Y. Let's suppose that inputs a and b generate 
output symbol e, and inputs c and d generate output symbol/. That is the nature of 
the channel, Q, which maps inputs to outputs. Using the mathematical definition of 
uncertainty, H, we can show that the information, I(p(X); Q), contained in each 
output symbol is 1 bit. That is, I(p(X); Q) = 1 bit/symbol. One bit of information 
effectively affords the receiver one "yes/no" decision. In other words, 1 bit of infor- 
mation halves his/her uncertainty regarding the input, X. This makes intuitive sense: 
After receiving the value /, he would consequently know that either c or d were 
input to the channel. Prior to receiving /, he could do no better than guess that the 
input was a, b, c or d. Upon receiving/, his uncertainty was halved. For this exam- 
ple, the information carried by an output symbol can be increased if the specificity 
of the channel, Q, is increased. In other words, it is desirable if the output describes 
the input with less ambiguity. If inputs a and b generate the output e, whilst c gener- 
ates / and d generates g, then we can arrive at an average information measure of 
1.5 bits/symbol. 

The simple communication channel of Fig. 17.3 is analogous to the above- 
mentioned process that converts high-resolution images of a small, moving target 
to phosphene images. In that process, the target varies in space, x(s), where s is a 
vector denoting space. The target is input to the image analysis scheme, Q, which, 
here, is a spatial filter, not a temporal one. The phosphene image, y(s), is analogous 
to an output symbol, and is rendered according to the output of the analysis scheme, Q. 
We developed this analogy in [6] by using the scanning model, that is, the APRL, 
as p(x(s)). See Fig. 17.4. 

We found that image analysis scheme Q2 imparts more information to the 
phosphene image than Q\ [6]. Specifically, Q2 imparts approximately 5 bits/ 
image as compared to QVs 1 bit/image. This improvement of 4 bits/image is the 
case when the target is presented to the phosphene image in a spatial pattern cor- 
responding to the scanning behavior found in [8]. Those scanning behaviors were 
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Fig. 17.4 The set-up of the numerical experiment involving a seven-phosphene image [6]. The 
high-resolution image (left) comprises a small, high-contrast target. The image is analyzed by the 
image analysis scheme (middle). The scheme comprises seven identical, Gaussian filter kernels; 
the circles indicate the first and second standard deviations of the kernels. Each kernel operates on the 
image and produces a response. Each response modulates the size of a phosphene comprising 
the phosphene image (right). In the example shown, the target activates two phosphenes since its 
location is "seen" by those two phosphenes, but no other phosphenes. For clarity, the phosphenes 
that are not activated are indicated with dashed outlines 



modeled by a bivariate normal with standard deviation equal to 0.25 times the 
phosphene-to-phosphene spacing. This finding provides some theoretical reasoning 
for subjects' accuracy in [8]: Q2 afforded better fixation and pursuit accuracy 
because it provides 4 bits/symbol more information to the phosphene image 
observer. 

Furthermore, our numerical experiment suggested that the image analysis 
scheme Q2 can be improved [6]. Specifically, the prediction is that, by using 
Gaussian kernels with standard deviation equal to 0.6 times the separation of 
phosphenes, performance will be further improved. This new scheme would 
make for approximately 8 bits of information in the phosphene image, assuming 
that subjects' scanning behaviors were unchanged. Indeed, the numerical 
experiment provides a prediction: that an image analysis scheme comprising 
Gaussian kernels with standard deviation 0.6 times the separation of phos- 
phenes should afford superior tracking as compared to Q2. Experimental work 
that examines this prediction, and similar predictions, will be the subject of 
future work. 

How are these measures of information to be interpreted? As discussed above, 
1 bit of information affords the phosphene image observer a single "yes/no" 
decision. For localizing a target, a single bit of information effectively allows the 
observer to divide the visual field into two halves and ask, "Which half of the 
visual field contains the target?" Therefore, when considering scheme Q\, 
affording the phosphene image observer 1 bit of information per image is akin 
to informing the observer whether the target lies within the left half, or the right 
half, of the area covered by the phosphene array or not. This being the case, we 
would expect the observer to deviate, on average, from the target by approxi- 
mately one-quarter of the diameter of the array. In contrast, scheme Q2, affording 
the phosphene image observer 5 bits of information per image is akin to informing 
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the observer which 1/32 of the area covered by the phosphene array contains the 
target. In this latter case, we would expect the observer to deviate from the target 
by less than that amount; the mean deviation would be reduced by approximately 
a factor of 4 (rather than 16, since the field is two-dimensional). 



17.5 Discussion 

We have reviewed two studies of ours that address image analysis, its use in micro- 
electronic retinal prostheses, and the perception of low-resolution images. These 
studies form the beginning of an approach that integrates theory and experiment 
and aims to better constrain the design of a prosthesis. Effectively, the approach 
seeks to answer the question, "How should high-resolution images be analyzed 
before rendering phosphene images?" 

In our psychophysical experiments, subjects fixated and pursued a small, moving 
target that was rendered on an array of phosphenes [8]. There, we showed that 
image analysis, which converts the high-resolution image of the moving target to 
the phosphene image, can indeed be used to improve subjects' performance. During 
trials, subjects scanned the phosphene array over the target, using some phosphenes 
in preference to others. We modeled this scanning using a bivariate function, and 
we termed this model the "artificial preferred retinal locus" (APRL). The experi- 
ments in [6] were numerical. There, we used the APRL in conjunction with various 
image analysis schemes and measured the information contained in the phosphene 
image. We found that the scheme affording subjects the best tracking performance 
imparted the most information to the phosphene image. Further, we found an opti- 
mal scheme of Gaussian kernels for image analysis which we predict would afford 
further improvements in performance. 

Our approach contributes to the existing literature on image analysis, its use in 
microelectronic retinal prosthesis, and the perception of low-resolution images. We 
have established an exchange between information theory and visual modeling, that 
is, the simulation of prosthetic vision using normal observers. This exchange allows 
for the design and implementation of image analysis schemes on the basis of quan- 
titative reasoning, and for those schemes to be verified via psychophysical methods. 
For example, our numerical experiment predicts that the optimal Gaussian scheme 
for fixation and pursuit involves kernels with standard deviation equal to 0.6 times 
the phosphene-to-phosphene spacing [6]. This prediction may be tested in a visual 
modeling experiment, prior to a test in actual implantees who are capable of per- 
forming simple visual tasks. Alternative approaches to image analysis often simply 
cite "biological inspiration." For example, an edge-detection scheme is often 
thought to be justified because edges are known to be of particular salience to the 
visual system. These approaches may have merit, but the design of the image analy- 
sis scheme seems arbitrary, and usually is not tested using visual modeling. 

It is important to consider the computational cost of an image analysis scheme. For 
our purposes, the operation of a kernel on an image involves A real multiplications 
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and A - 1 real additions, where A is the area of the kernel in pixels. Therefore, the 
relative cost of image analysis scheme Q2 is proportional to the area of the Q2 kernel 
divided by that of the Q\ kernel. In many image processing applications, Gaussian 
kernels are restricted to a circular support with a radius of three standard deviations. 
Our circular averaging kernels (Ql) had a diameter equal to the separation of phos- 
phenes. Therefore, the ratio of areas of these kernels is 3.24. That is, in our tracking 
study [8], the computational cost of using Q2 was 3.24 times that of Ql. 

So far, our approach involves models of scanning that vary in space, but not 
time. However, scanning is likely to be better described by models that are spa- 
tiotemporal. A spatiotemporal model of scanning would describe not only which 
phosphenes were used in preference to others, but how the outputs of many phos- 
phenes were used in combination over time. In other words, a spatiotemporal scan- 
ning model would describe how subjects tended to sweep the phosphene array 
across the high-resolution target. Our psychophysical data suggest that the temporal 
nature of scanning is important (see also [3]). For example, the image analysis 
scheme QQ compelled subjects to use nystagmus-like scanning, rapidly moving the 
array back and forth across the underlying target. Rather than using the information 
contained in the phosphene image at a single instant, subjects integrated the phos- 
phene array activity over short periods, and used that integrated information to 
guide behavior. Developing scanning models, that is, APRLs, to include second- 
and higher-order statistics is the subject of ongoing work. 

Our approach concerns visual fixation and pursuit. In the future, we aim to 
extend the approach to include other tasks, such as reading. To do so, our psy- 
chophysical experiment [8] could be modified to involve the identification of com- 
monly used words, as opposed to the tracking of a small, moving target. In this new 
experiment, subjects would employ scanning behaviors that were specific to reading. 
Then, images of these commonly used words could be used as stimuli in the 
numerical set-up of [6], and an image analysis scheme could be tailored to these 
images. Overall, it is likely that different image analysis schemes would apply to 
different visual tasks. For example, a Gaussian scheme, like Q2, may be suited to 
tracking a small, moving target, but some other scheme involving some other class 
of kernels may be better suited to reading, for example, oriented Gabor functions. 



17.6 Conclusion 

We have discussed our approach to image analysis, microelectronic retinal prosthe- 
ses, and the perception of low-resolution images. We believe that this approach can 
be used to help constrain the design of an implant. The approach is analogous to the 
acoustic modeling of cochlear implants which involves normally hearing listeners. 
That approach has made important contributions to the improvement of clinical 
outcomes in cochlear implantees since 1990. We hope that our visual modeling 
approach, and developments thereof, will ultimately contribute to improved clinical 
outcomes in retinal implantees. 
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Chapter 18 

Simulations of Cortical Prosthetic Vision 



Nishant R. Srivastava 



Abstract Cortical stimulation for restoring vision presents researchers with many 
challenges and questions. The extent of the human visual cortex varies up to 50% 
from one individual to another, cortical folding and sulci limit the area of implan- 
tation, and surgical difficulties make it difficult to implant electrodes to produce 
phosphenes in the whole visual space. Researchers are faced with question such as: 
which electrodes to use - surface electrodes that are easy to implant or intracorti- 
cal fine-metal electrodes that have lower current requirements and have five times 
better resolution? How many phosphenes will be enough to give limited, but useful 
vision? How will cortical physiology affect phosphene maps? Will percepts be dis- 
tinct dots or complex in nature? What will be the long term response to stimulation? 
Will the brain adapt to seeing through dotted images? Some of these questions can 
be answered by conducting human psychophysical tests. 



Abbreviations 

f MRI functional Magnetic resonance imaging 

LGN Lateral geniculate nucleus 

VI Striate cortex or primary visual cortex 

V2 Prestriate cortex or secondary visual cortex 

V3 Third visual complex 

18.1 Introduction 

In the 1990s, Cha et al. simulated arrays varying from 100 electrodes (10x10 
arrays) to 1,024 electrodes (32x32 arrays), represented by small dots in a video 
display mounted on ski goggles to test the requirements of a cortical prosthesis 
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device [4-6]. The images were captured by a head mounted video camera covered 
by a perforated mask. They concluded that, with 625 dots (25 x 25 array) in a visual 
field of 1.7°, a visual acuity of 20/30 and a reading speed of 100 words of paragraph 
text per minute could be achieved. They found that 625 or more dots, with a field 
view of about 30°, allowed normal walking speed through a maze with obstacles. 

For these tests, they used identically-sized simulated phosphenes on regularly- 
spaced grids. Cortical stimulation experiments, however, have shown that, for a 
cortical prosthesis, a regular grid structure is not a true representation of the phos- 
phene map. The maps will vary depending upon the area in the visual cortex used 
for implanting the electrodes and the type of electrodes used. For conducting proper 
simulation studies, simulated phosphene maps have to correspond to the map gen- 
erated as per the targeted electrode location and the type of electrode used. 

Some groups might target the medial surface for implantation, and a few groups 
might target the lateral surface or a combination of electrodes on the lateral and the 
medial surface. Every research group that is targeting the cortex for electrode 
implantation will have to generate an estimated percept map depending on their 
choices of electrodes and array location, and use this map to guide expectations for 
this device's performance. To generate this map, the representation of visual space 
on the cortex, cortical structure, and the corresponding biological responses have to 
be understood. 



18.2 Representation of Visual Space on the Visual Cortex 

One of the first published visual maps by Holmes shows representation of different 
visual fields on calcarine cortex, with a linear relationship of visual space to the 
cortex [12]. These maps were later modified by Horton and Hoyt [13]. The modi- 
fied map shows the horizontal meridian running at the base of the calcarine fissure 
with iso-eccentricity contours from 2.5° to 40°. This map shows that the space fol- 
lows a logarithmic representation till 40° of eccentricity, with the foveal area of 
visual field represented on a larger area of cortex. Recent experiments have sup- 
ported these modified maps [17, 18]. This logarithmic representation of visual 
space on the visual cortex is known as cortical magnification. 

In an fMRI study by DeYoe and colleagues, a consistent retinotopic organization 
was observed on responsive visual cortex both medially and ventrally [7]. The 
foveal representation was located posteriorly, near the pole, and greater eccentrici- 
ties were represented anteriorly on the surface. Responses observed to visually 
expanding checkered rings extended from the collateral sulcus on the ventral sur- 
face, crossed the calcarine fissure and passed dorsally out onto the exposed lateral 
surface. The responses did demonstrate cortical magnification. The data also 
showed that as the eccentricity increased, there was an anterior progression of acti- 
vation along with alteration of visual field meridian at transfer from one visual area 
to other. In another fMRI study by Levy and colleagues, it was observed that the 
visual cortex has a hierarchical organization that begins with the precise visual field 
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maps in VI, V2 and V3, which degrades on the lateral and ventral regions [16]. The 
lateral and ventral regions contain coarse eccentricity maps with crude representa- 
tions of the polar angle. It has been observed that visual space is represented in 
many visual clusters. Wandell and colleagues reported observing nine human visual 
field map representations [22]. These observed visual field maps preserved spatial 
structure. 

Any electrode implantation will have to consider the logarithmic nature of these 
representations. Electrodes implanted near the occipital pole will cover a very small 
visual area corresponding to a few degrees of eccentricity, and phosphenes corre- 
sponding to electrodes on the lateral surface may be perceived more eccentrically, 
but will lose angular specificity. If the electrodes lie in an area with multiple cluster 
maps, then the position of the phosphenes in visual space will be almost random. 
Any psychophysical study designed to estimate the performance of a cortical pros- 
thesis device will have to consider these factors. 



18.3 Cortical Stimulation Studies 

Brindley and Lewin placed eighty 0.64 mm 2 -platinum electrodes between the 
medial surface of the occipital pole of the right cerebral hemisphere and the falx 
cerebri of a 52-year-old blind female patient [1—3]. They compared the observed 
phosphene maps with the Holmes map and found correlations between the two. 
This experiment was performed in 1968, when the revised map by Horton and Hoyt 
was not available. If the map published by Brindley and Lewin is compared with 
the map published by Horton and Hoyt, it is observed that the phosphene map does 
show cortical magnification. The phosphenes at the periphery of the Brindley and 
Lewin phosphene map were larger in size, which might be result of cortical magni- 
fication. A few irregularities were observed, such as an electrode placed close to a 
certain group of electrodes produced a phosphene away from the phosphenes cor- 
responding to the group. 

Dobelle and Mladejovsky published phosphene maps generated from surface 
electrodes placed on the right medial surface of a patient [8, 9]. These maps show 
discrepancies from the expected responses when compared to a logarithmic visual 
space map, both in terms of eccentricity of phosphenes observed, and polar angles 
expected from published visual maps. 

Dobelle and colleagues published another set of results with 64 platinum disk 
surface electrodes implanted 3 mm apart on the medial surface of right occipital 
lobe [10]. The phosphenes followed the expected cortical magnification on visual 
space. A set of electrodes placed in a line close to calcarine fissure was expected to 
define the horizontal meridian; instead, it was almost perpendicular to it, close to 
the vertical meridian. It was hypothesized that the electrodes crossed into the V2 
area hence showed the mirror image of the expected response from VI. This shows 
that the phosphenes might be produced by stimulating higher areas of extrastriate 
cortex. On the lateral surface, the boundaries of VI, V2 and V3 are not clearly defined. 
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Even with a very small area of VI exposed on the lateral surface, though, studies 
have still shown the generation of phosphenes [14, 15, 19]. 

Schmidt and colleagues stimulated the lateral surface of visual cortex using 
intracortical electrodes [19]. The electrodes were placed approximately up to 
22 mm away from the occipital pole on the lateral surface. Most of the phosphenes 
mapped were within 30° eccentricity, with few phosphenes up to 40° eccentricity. 
Few of the phosphenes were observed at the eccentricities and polar angles 
expected when compared to the visual maps discussed earlier, and a few phos- 
phenes did not exhibit either the expected eccentricity or the polar angle. 

Lee and colleagues stimulated the occipital cortex and the adjacent cortices 
using surface electrodes in 23 epilepsy patients [15]. The experiment shows that as 
electrodes are implanted away from the occipital pole, more anteriorly towards the 
frontal cortex, the response to stimulation varies from simple form phosphenes, to 
intermediate responses like triangles and diamond shapes, to complex responses 
like observations of color and evoking movement percepts. The initial 1-2 cm 
(approximate) from the occipital pole show simple responses, and 2-3 cm (approxi- 
mate) show intermediate responses. If electrodes are placed on the lateral surface, 
they should be limited to a distance of 3 cm from the occipital pole. 

Kaido and colleagues investigated retinotopic maps on the lateral surface of the 
occipital cortex in humans [14]. The researchers observed phosphenes of up to 80° 
eccentricity, stimulating up to 40 mm anterior to the occipital pole, on X-ray scale, 
which shows that the whole lateral surface of occipital cortex might generate pho- 
sphenes. Polar angles were preserved in a coarse manner. If electrodes are implanted 
too far anterior from the occipital pole, however, complex forms might be observed, 
as in stimulation studies by Lee and colleagues. [15]. 



18.4 Variability in Occipital Cortex 

Stensaas and colleagues studied primary visual cortex of 52 hemispheres and 
found the average total area to be 2,134 mm 2 [21]. The average striate cortex 
exposed on all four surfaces was 689 mm 2 , about 33% of striate cortex, and the 
other 67% of striate cortex (average 1,445 mm 2 ) was buried in fissures [21]. For 
inter-electrode distances of 3 mm, we might be able to place only 60-80 elec- 
trodes on VI [1-3, 8-10, 21]. It was found that, on average, only 3% (55 mm 2 ) 
of primary human visual cortex extends to the occipital surface of the brain. For 
the average exposed VI area of 55 mm 2 , if intracortical electrodes are used with 
inter-electrode distances of 0.5 mm, it might be possible to place 200 electrodes 
[19]. The variability in the visual cortex within individuals will affect cortical 
implantation. If surgical methods are developed to implant electrodes over the 
complete exposed striate cortex, then the implanted electrode numbers might 
vary by 30%. 

Dougherty and colleagues used data from fMRI and prepared 2-D flattened 
representations of the cortical manifold [11]. Using these flattened maps they 
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calculated VI, V2 and V3 sizes. The left hemisphere of VI was about 200 mm 2 
larger than the right hemisphere of VI. Mean areas of V2 and V3 in left and right 
hemispheres were not significantly different. Dorsal VI, V2 and V3 regions were 
found to be larger than ventral VI, V2 and V3 regions. This will allow creating 
more phosphenes by implanting electrodes on the dorsal surface than on the ventral 
surface. They also found that the surface area of V2 representing eccentricities of 
2°-12° was roughly 75% of that of VI, and that of V3 was only 56% the size of 
Vl's corresponding area. They hypothesized that V2 either receives only a portion 
of the VI output, or it has a more efficient representation of VI output. They found 
that cortical magnification does not differ significantly between left and right hemi- 
spheres for VI, V2 and V3. 

These results guide us to estimate the number of the electrodes which can be 
implanted on the targeted area and construct a phosphene map for it. These studies 
show that cortical surfaces can vary about 50% from one individual to other. Hence, 
the number of electrodes that can be implanted can vary up to 50%. This has a 
direct impact on the number of simulated phosphenes that should be used for psy- 
chophysical studies. Such psychophysical studies will have to consider this varia- 
tion when generating dotted images, and for every placement, will have to consider 
a dropout rate of phosphene from 25 to 50%. 



18.5 Phosphene Map Estimation 

If electrodes are implanted on the medial surface of striate cortex, a phosphene 
map as shown in Fig. 18.1 can be expected [1-3, 8-10]. The phosphene size 
increases at higher eccentricities, and the distance between phosphenes increases, 




Fig. 18.1 VI area on the medial surface shown by dots as the electrodes with the corresponding 
expected phosphenes in the visual space. Lateral blank visual space corresponds to the area buried 
in the calcarine fissure 
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Fig. 18.2 The visual map expected with placement of electrodes on the medial wall of the V 1 , 
V2 and V3 areas. Note the additional phosphenes, mostly in the lateral visual field, as compared 
with Fig. 18.1; V2 and V3 phosphenes are largely intermixed 



reflecting the distortion due to cortical magnification [1-3, 8-10]. The gap 
shown on the lateral visual field corresponds to the two-thirds of area VI that 
lies within the calcarine fissure. This area is inaccessible with existing surgical 
techniques for both surface and the intracortical electrodes. It might be possible 
to generate phosphenes on the lateral visual field by stimulating V2 and V3, 
which have representations similar to that of VI. If electrodes are implanted on 
V2 and V3, along with VI, the mirror image correspondence of VI to V2 and 
V2 to V3 might help to generate phosphenes on the lateral surface as shown in 
Fig. 18.2. 

Schmidt and Kaido have shown the generation of phosphenes over a wide region 
of visual space, while stimulating the lateral surface, and concluded that this area 
can be used to create phosphenes for a cortical visual prosthesis [14, 19]. From the 
study of Lee and colleagues we observe that the limitation of area for electrode 
placement area on the lateral surface area is about 3 mm from the occipital pole to 
get a simple percept [15]. The fMRI study by DeYoe shows that for a 3 cm radius 
from the occipital pole, phosphenes might be observed throughout the central 25° 
of the visual field [7]. The expected phosphene map is represented in Fig. 18.3. This 
figure is derived from an fMRI study, but as observed with intracortical electrode 
experiments, few phosphenes might be observed in 40°-45° [19]. If it is assumed 
that 50% of phosphenes from this hypothesized map are between 25° and 45° 
eccentricity, this will give us a map in which 50% dots of phosphenes are dropped 
from the initial 25° of the eccentricity map in Fig. 18.3, and redistributed across 
25°-45° eccentricity, giving us a map as shown in Fig. 18.4. 
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Fig. 18.3 Visual map showing phosphenes within 25° eccentricity expected with electrodes on 
the lateral surface area in a radius of 3 cm from the occipital pole. This map corresponds to fMRI 
studies 



Phosphene map if phosphene 
appear upto 40-45 degrees of 
icily in lower quadrant 




Fig. 18.4 Visual map with phosphenes in 40°^15 o eccentricity. This map corresponds to observa- 
tions by Schmidt in 1996, in which few phosphenes were observed at about 40°^t5° [19]. If we 
get this scenario where the phosphenes are generated up to 40°^15° then we will have a larger field 
of view than we expect from fMRI studies and lower density of phosphenes 
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18.6 Psychophysical Studies with the Estimated Maps 

In the field of visual prosthetics, very limited psychophysical studies have been 
conducted to judge the performance of a cortical prosthesis. After the studies by 
Cha and colleagues, many psychophysical studies were conducted targeting other 
areas of vision restoration or for judging different image processing schemes, but 
detailed psychophysical studies addressing cortical structure and biological limita- 
tions have been missing [4-6]. In 2007 psychophysical tests were conducted at 
John Hopkins University by Srivastava and colleagues for testing the expected 
response of a cortical prosthesis, being developed at the Illinois Institute of 
Technology, Chicago, using 650 intracortical electrodes targeting the dorso-lateral 
surface [20]. Tests were conducted on five volunteer subjects, three men and two 
women, using a dotted phosphene map similar to Fig. 18.3. Individual variations in 
the cortex and failure of some electrodes to elicit a phosphene were also incorpo- 
rated in these studies. 

Area and layout of the visual cortex may vary by up to 50% between individuals 
[7], hence the anatomical area for electrode implantation will similarly vary, hence 
in few patients the area available for electrodes might be less than the average. In 
addition, some electrodes, or stimulation sites, might fail during the surgical proce- 
dure itself, or during long-term implantation. To study the effect of fewer phos- 
phenes than might be expected from the electrode count, dropout effects were 
included in our studies. Performance was judged under 0% dropout, 25% dropout 
and 50% dropout conditions. Three different tasks were selected to observe if, with 
these limited-field phosphene maps, persons can adapt to perform different tasks of 
eye-hand coordination and mobility. These tasks were judged to be representative 
of the basic tasks required to be done in daily day-to-day life. These experiments 
were conducted using eye-tracking to simulate the normal movement of phos- 
phenes in visual space. 

The purpose of the first two psychophysical experiments were to determine the 
subject's ability to perform detection by counting the white fields on a checker- 
board and eye hand coordination by placing black checkers on the white fields of 
the checkerboard. The accuracy of counting and placing along with the time taken 
to complete the task were used to judge performance. Subjects were able to inspect 
the board by scanning the head-mounted camera in parallel with the board, or 
change their viewing angle by tilting their heads. The decision by the subjects for 
scanning the boards was intuitive for each individual. By giving the subject the 
ability to scan the board, spatial and temporal integration was achieved. The experi- 
mental setup is shown in Fig. 18.5. The leftmost image shows the phosphene map 
for the dorso-lateral surface with 650 electrodes implanted on a region of 3 cm 
radius area, with occipital pole as the center. The center image shows the checker 
board used, and the rightmost image shows the dotted image seen by the test sub- 
jects in a headset with eye tracking. 

Counting time for counting all the white fields and the number of fields reported 
by the subject were recorded for each trial. The significant factors effecting the 
study were individual differences in subjects (F=25.29, /?<0.0001), increase 
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Fig. 18.5 Checkered board as observed through a phosphene-like map 



in practice (F=23.58, /?< 0.001), dropout (F=8.41, p = 0.0040) and increase in 
complexity (F= 50.75, p<Q.Q0l). The dropout had a weak significance compared 
to the other factors showing that the effects of dropout could be overcome by prac- 
tice. The same analysis was done for each individual subject, and practice was the 
significant factor in all subjects except the low vision subject. 

Similar to counting time, placing time was also recorded. The significant factors 
effecting placing timing were variation due to individuals (F= 14.02, /?<0.0001), 
increase in practice (F=30.12, p<0.001), the dropout (F=9.03, /? = 0.0040) and 
the number of white fields on the board (F= 109.42, p< 0.001). Dropout had a 
weak significance compared to the other factors. For individual analysis, all the sub- 
jects show a significant reduction in task time with increasing practice. Three 
subjects showed an increase in the time with an increase in the dropout level, but 
with a modest significance level. This shows that the practice effect dominates 
over the dropout effect, and a decline in performance due to dropout can be over- 
come by practice. 

In the third experiment, subjects' ability to recognize a pathway and orient oneself 
to follow it, without memorizing or recall, was judged by observing the performance 
of the subjects' maneuvering in virtual mazes. While for the counting and placing 
tasks, the subjects were observing the checkerboards with a camera, and could there- 
fore adjust the level of detail by changing their viewing distance and angle, the full 
scene of the virtual mobility experiment, was imaged onto the phosphene map, and 
subjects were trained to use a game controller to change their vantage point, as if 
moving through virtual space. Thus, depending on the task, the acceptance angle of 
the camera or other image source may have to be adjusted by the subject to get an 
optimum view. This can also be achieved by selecting the part of the captured image 
to be presented in the phosphene map. This is shown in Fig. 18.6 where the virtual 
maze is shown on the left, which was made to fit on the central phosphene map by 
adjusting the image scale in the graphics processor. The resulting dotted image is 
shown in the right image. 

The effect of learning on the performance was observed with increased practice 
of the experimental task. The time to complete each maze and the number of way- 
finding errors were recorded. The factors affecting the experiment were individual 
differences (F=34.21,/j<0001), increase in practice (F= 90.04, p< 0001), and the 
dropout (F=6.06, p=0.0189). Dropout had a low significance, similar to the counting 
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Fig. 18.6 Phosphene-like image of virtual maze 



and placing experiments, and a separate analysis checking the effects within indi- 
viduals yielded practice as the only significant effect. Other factors and interactions 
had no statistical significance. 

The results from these experiments demonstrated that even with a limited number 
of phosphenes in one visual hemifield, with limited eccentricity in visual space, it 
is possible to attain a level of proficiency with which prosthesis wearers can per- 
form simple tasks, albeit with practice. 

A significant finding from these studies was that the degradation in the image, due 
to a lower number of phosphenes (higher dropout), could be largely overcome by an 
increase in practice and use by the tested subject. This result can be significant for 
individuals with a small area of visual cortex available for electrode implantation, who 
may still be candidates for prosthesis implantation. This result also is reassuring for 
researchers who worry about the failure of electrodes, or neuronal stimulation sites, 
during surgical implantation and over the lifetime of a prosthesis, and what effect this 
may have on performance. These results also may be of value to visual prosthesis 
researchers using other substrates, such as the retina, optic nerve, or lateral geniculate 
nucleus (LGN). As an indication for what they can expect from such devices. They 
may also provide a guide to vision scientists conducting simulation studies, in regard 
to how biological response and surgical difficulties might affect their results. 

As human studies are conducted and more real maps are obtained, similar 
experiments might be repeated using more realistic maps, and potentially providing 
better predictions of performance. The experiments conducted by Cha and col- 
leagues gave 625 electrodes for getting a limited sense of vision. In the experiments 
mentioned in this chapter by Srivastava and colleagues, it was observed that even 
with 325 electrodes, and an incorporated estimate of cortical distortion in the pho- 
sphene maps, the subjects learned and adapted to performing the basic tasks [20]. 

Another observation of this study was that producing a large number of phos- 
phene in a limited visual field might not be helpful, since many of them might 
potentially overlap, and the effective result would not be a major improvement in 
recognition ability. It would be more helpful to create distinct phosphenes on larger 
visual area, which might require development of improved surgical techniques. 

The relatively crude and limited information provided by the cortical visual 
prosthesis under development by various research groups, if successful, might help 
blind subjects obtain some assistance for conducting simple tasks in daily life. 
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Chapter 19 

Phosphene Mapping Techniques for Visual 

Prostheses 

H. Christiaan Stronks and Gislin Dagnelie 



Abstract Mapping of the visual world onto the visual system occurs in a highly 
ordered manner, yet with substantial interindividual variability. Since the retinal 
map of the scene at the photoreceptor level is fully determined by the optical pro- 
jection of the eye, it is likely that a proximal map generated by a retinal prosthesis 
closely adheres to the same geometric projection. Once the nerve signals enter the 
optic nerve, this orderly map is redistributed, and while maps at more proximal 
levels still follow general rules, special mapping techniques in individual LGN 
or cortical prosthesis recipients will be required to allow reconstruction of spatial 
relationships in the outside world by means of a disorderly array of phosphenes. 

This chapter provides an overview of mapping techniques that have been used 
in a number of laboratories; discuss the strengths and weaknesses of each; and sug- 
gest ways in which various techniques can be combined. 



Abbreviations 



HMD Head mounted display 

MDS Multidimensional scaling 

TMS Transcranial magnetic stimulation 



19.1 Importance of Mapping 



Ever since researchers first started eliciting phosphenes in blind patients through 
electrical stimulation there has been a need to specify the location of the phos- 
phenes in the visual field. Over 30 years ago phosphene mapping was defined by 
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Everitt and Rushton as "determining the position of each phosphene in the visual 
field" [13]. Phosphene mapping is an important step in determining the functional- 
ity of visual prostheses. Mapping phosphenes allows the characterization of how 
evoked phosphenes by stimulation of the different electrodes of a visual prosthesis 
cover the visual field. After the phosphene map is obtained, clinical fitting proce- 
dures can be applied to adjust the visual input stage (e.g., the video image) and 
visual processing strategies to provide the prosthesis wearer with a proper percept 
representative of the outside world. 

In general, there are two different approaches to obtain a phosphene map; abso- 
lute and relative phosphene mapping. Absolute maps describe the position of 
phosphenes in absolute coordinates in the field of view, while relative maps pro- 
vide information about the spatial relationships between phosphenes, in terms of 
distance and angle. Both methods have advantages and disadvantages, as discussed 
below. 

Retinal prostheses likely yield predictable phosphene maps, since the represen- 
tation of the outside world on the retina (i.e., the retinotopical organization) is 
determined by simple geometry and is constant across subjects [12]. Nevertheless, 
retinal neural organization has been shown to change during prolonged periods of 
visual impairment (e.g. [15, 25] and Chap. 3), so phosphene mapping might still 
prove important in patients with long-term vision loss. Since retinotopy is largely 
preserved in the optic nerve, implants in the optic nerve are likewise expected to 
yield predictable phosphene maps, although the accuracy and stability of such a 
map will depend on the ability to precisely position the electrodes, due to the high 
density and thin caliber of the nerve fibers. 

Phosphene maps obtained in cortical prosthesis users are relatively unpredict- 
able, since multiple maps of the visual field are represented in different cortical 
areas which may cause widely spaced electrodes to evoke phosphenes in different 
cortical areas, where they may or may not fall in the same area of the visual field 
[11]. Moreover, the presence of sulci and gyri in the cortex may lead to unexpect- 
edly large distances between phosphene locations in the visual field. Finally, 
cortical organization differs from person to person and, more importantly, the 
functional organization in longer-term visually impaired individuals may be sub- 
stantially different from normal- sighted people due to the plasticity of the visual 
cortex [22]. Therefore, phosphene mapping will be especially important in corti- 
cal prosthesis recipients. 

This chapter deals with various phosphene mapping techniques. Comparable 
mapping techniques will be discussed together and will be presented in roughly 
chronological order, starting with the earliest report on mapping techniques from 
Brindley and Lewin. Wherever possible, comparable studies (e.g., cortical and 
retinal prostheses, simulation studies etc.) will be discussed together in the text. 
The chapter concludes with results from our laboratory, a short overview of 
phosphene mapping methods, and suggestions will be made which methods to use 
in different situations. 
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19.2 Early Absolute and Relative Mapping Procedures 

in Subjects with Cortical Prostheses: Pointing Techniques 

Mapping of phosphenes already proved to be highly informative during the 
pioneering work of Brindley and Lewin. They acknowledged the importance of 
absolute and relative phosphene mapping in the 1960s. In one of their studies they 
chronically implanted a subject with no functional vision with an array of 80 sub- 
dural extra-cortical electrodes [3]. Absolute maps were obtained by letting the 
subject point towards the perceived phosphene with the left hand in a hemispherical 
bowl with a radius of 0.59 m, i.e., approximately at arm's length. The right hand 
grasped a small knob inside the bowl for tactile reference. Relative maps were 
obtained by sequentially stimulating two electrodes and asking the subject to 
describe the spatial relations between the two phosphenes, such as distance and 
compass angle. 

The early experiments of Brindley and Lewin already showed that phosphene 
maps were not a simple reflection of the electrode array projected into the visual 
field of the subject. Rather, phosphene maps roughly corresponded to the classical 
cortical maps constructed by examination of gunshot victims of WWI (e.g. [17]), 
that showed the nonlinear projection of the visual field onto the cortical surface. 
Their experiments also showed that these phosphene maps were not very regular. 
Phosphenes lying in a straight line in the visual field could be evoked by electrodes 
lying in a triangular configuration on the cortical surface. It was also shown that 
activation of distant electrodes could result in phosphenes overlapping in the visual 
field. In addition, stimuli delivered well above threshold by a particular electrode 
could result in additional phosphenes being elicited in distant locations. These early 
experiments strongly indicated the need for proper phosphene mapping, since pho- 
sphene configuration may differ substantially from electrode organization. 

Dobelle and Mladejovsky [10] performed similar experiments in acute sessions 
on normally sighted patients undergoing occipital lobe surgery. One patient 
received a sub-chronic implant for a period of IVi days and most of the data were 
obtained from this subject. Phosphene mapping was performed by letting the sub- 
ject point to where the phosphene was perceived. Maps were then created by drawing 
the phosphenes in a visual field map. Relative maps were obtained by asking the 
subject to describe how different phosphenes interrelated. Though the phosphene 
mapping techniques are not discussed in detail, the authors provide detailed analyses 
of the phosphene maps they obtained and include a critical discussion on the 
mapping techniques employed. 

In conjunction with the classic cortical maps, they found that for a given inter- 
electrode distance, phosphene spacing varied depending on the area of the cortex 
being stimulated. Phosphenes close to the center of the field of vision (e.g., elicited 
by stimulation of electrodes near the occipital poles) were usually closer together 
than those in the periphery. Moreover, when the electrode array spanned a cortical 
fissure (sulcus), a gap between phosphenes in the visual field was observed, which 
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could be explained by the fact that the electrodes did not penetrate deep enough to 
stimulate the cortical tissue within the sulcus [10]. 

The authors also recognized some of the most important advantages and disad- 
vantages of the phosphene mapping techniques employed both by them and by 
Brindley and Lewin. While absolute mapping provides the scale of the map, relative 
mapping provides the detailed interrelationship of phosphenes [3], making both 
techniques complementary. Relative mapping is more time consuming and may 
therefore not be suitable for acute testing in the operating room. Nevertheless, 
relative mapping using (near-) simultaneous phosphene presentation may be pref- 
erable over absolute maps obtained by sequential activation of different electrodes, 
since phosphenes move with eye-position, making absolute localization in the 
visual field difficult. Another general disadvantage of (absolute) mapping by 
pointing is that phosphenes elicited by different electrodes may be too close 
together to be resolved properly due to inaccuracies in pointing, especially in blind 
subjects who have no visual feedback [10, 26]. 

The pointing method described in the 60s by Brindley and Lewin was applied 
in much the same way by Gothe et al. [16], who investigated cortically evoked 
phosphenes by means of transcranial magnetic stimulation (TMS). TMS is a 
method to affect cortical activity, and sometimes evoke phosphenes, by electro- 
magnetic stimulation through the intact scalp and skull. Gothe et al. instructed 
sighted subjects and individuals with residual vision to use a laser pointer in a 
room with dimmed lights to indicate the position of phosphenes onto a semicircular 
screen that was placed 120 cm before the subject and extending 33° on each side. 
Subjects without residual vision were instructed to point in the direction of the 
percept. 

They found that the number of cortical locations from which phosphenes could 
be evoked increased with the amount of residual vision. In normal-sighted subjects, 
stimulation of all areas of the occipital lobe yielded phosphenes, while in totally 
blind subjects only 20% proved responsive. These results were somewhat surprising, 
since Brindley and Lewin reported that activation of almost all of their 80 elec- 
trodes yielded perceivable phosphenes [3]. 



19.3 The Computer Era: Refining the Pointing 
Method of Phosphene Mapping 

Following the pioneering studies of Brindley and Dobelle, relative phosphene map- 
ping was improved during the 1970s when computers became available. Hand- 
made maps could be digitized using the relative coordinates of each phosphene [9] . 
Everitt and Rushton [13] proposed a method to combine all the available data in a 
patient by digitizing and pooling relative maps and using an iterative "best fitting" 
procedure to obtain a reliable relative phosphene map. They were actually able to 
use these maps to present figures and letters by direct electrode stimulation which 
yielded patterns that were recognizable by the subject. 
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Dobelle and associates worked out a fully computerized protocol to overcome 
the problems of eye drift and inaccuracy of pointing [26]. Subjects were presented 
with pairs of simultaneously evoked phosphenes, minimizing the effect of eye-drift. 
One electrode was stimulated for 1 s, the other 3 s and the subject was asked to 
report the spatial relationship between the "short flash of light" and the "long flash 
of light". The subject entered the relative position of two phosphenes into a com- 
puter through two key presses on a touch-tone telephone pad, using "5" as a reference 
key. Thus, "1" encoded "above and left", and "2" "directly above" etc. By mapping 
different phosphene combinations according to their relative X and Y coordinates, 
the authors were able to construct relative phosphene maps. 

This procedure was applied on a male patient blinded by a gunshot wound who 
was subsequently chronically implanted with a subdural 64-electrode array on the 
occipital cortex (knows as the striate cortex, VI, or area 17). The computerized 
procedure not only resulted in a relative phosphene map, it also enabled the authors 
to construct an accurate and detailed layout of the cortical surface under the elec- 
trode array [11]. With this map they could accurately predict where sulci were situ- 
ated under the electrode array and even how deep a given sulcus was by determining 
the magnitude of the shift in phosphene location of adjacent electrodes. Boundaries 
of the striate and peristriate cortical areas could be identified by a reversal in phos- 
phene direction when adjacent electrodes were stimulated (these areas contain 
reversed maps of the visual field). The calcarine fissure along the medial wall of the 
occipital lobe - known to separate cortical areas representing areas above and 
below the horizontal meridian - could accurately be identified by a sudden shift in 
adjacent electrodes evoking phosphenes in the upper and lower visual field. 
Furthermore, electrodes that could elicit phosphenes in different visual field loca- 
tions, depending on the current level, were found to lie alongside sulci: the inter- 
vening portion of the visual field projected to the portion of cortex in the sulcus, 
and could therefore not be activated with the surface electrodes used at the time. 

The methods for phosphene mapping essentially did not change much during 
the decades following Dobelle's work. Bak and colleagues [1] mapped size and 
absolute position of phosphenes by instructing intracortically stimulated (sighted) 
subjects during acute recordings to fix their gaze and point with their finger on a 
white screen with calibrated markers where the phosphene was perceived. Later, 
for absolute phosphene mapping in a chronically implanted subject, they used a 
dart board with 12 sectors and five annular zones for tactile feedback. The subject 
was asked to place a dart at the location of the phosphene while keeping her gaze 
fixed. For relative mapping they used the computerized method from Dobelle and 
associates and improved the resolution by deploying a joystick that could detect 
16, instead of eight, relative angles [28]. The latter method was supplemented with 
verbal information to incorporate spacing between phosphenes. 

The Brussels group of Veraart published several papers in which they mapped 
phosphenes by letting a subject point to the evoked phosphenes. Phosphenes were 
evoked with a four-electrode optic-nerve prosthesis that was chronically implanted 
in a subject suffering from retinitis pigmentosa without useful light perception 
[8, 29, 30]. The four electrodes were positioned around the right optic nerve. 
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Absolute phosphene mapping was performed using a method very similar to that 
described by Brindley and Lewin 30 years earlier. The chronically implanted subject 
was instructed to point to the location of the perceived phosphene in a hemisphere 
with a radius of 0.45 m. While the task was performed, the volunteer's head was 
steadied in front of the hemisphere using a frame that provided support for the fore- 
head, chin and parietal skull. The subject's index finger was placed on the fixation 
point (a disc in the center of the hemispheric surface) as a proprioceptive reference. 
The subject was instructed to fix her gaze at the (unseen) fixation point and eye 
movements were recorded with a camera. Furthermore, electro-oculograms were 
assessed to monitor eye movements. To help the subject identify phosphenes, elec- 
tric stimuli were preceded and followed by a tone. The fingers of the right hand were 
used to indicate the perceived phosphene as a shape on the hemisphere. Various 
phosphene characteristics were recorded such as position, dimensions and motion. 

Interestingly, dependent on the exact stimulation parameters, 64 different phos- 
phenes varying in shape and size could be elicited. More importantly, the phosphenes 
covered a visual angle of about +35° to -50° vertically and -30° to +30° horizontally, 
despite the fact that the implant contained just four electrodes. 

Subsequent studies on this subject were performed using phosphene mapping 
methods very similar to the first study. The subject's gaze was steadied and moni- 
tored in the same way and phosphenes were localized similarly using the pointing 
hemisphere. Mapping was performed by an observer who copied the azimuth and 
elevation coordinates from the hemisphere with the aid of meridians and parallels 
traced on the hemisphere. These data were then transferred to a digital database in 
which phosphenes were described as pixels with 1° resolution. Phosphene area was 
defined as the number of pixels within the phosphene. 

Again, like in the previous study, phosphenes covered a large area of the visual 
field (from -30° to +30° horizontally and +20° to -50° vertically), despite the pres- 
ence of only a very limited number of electrodes. Exact location depended mainly on 
the electrode position and current level. Each quadrant of the visual field was mostly 
accommodated by one electrode and higher current levels evoked phosphenes closer 
to the fixation point. Position was also influenced by duration, number of pulses and 
pulse rate of the applied pulse trains. Phosphene size and luminosity did not clearly 
depend on any parameter, but tended to increase at higher stimulus levels. No relation 
was found between stimulating condition and phosphene color or "texture" [8, 30]. 

In a later publication they increased the total number of individually addressable 
phosphenes in the phosphene map to 109, excluding the additional "ghost" phos- 
phenes that appear at higher current [2]. By mathematically fitting model equations 
relating phosphene location to the afore-mentioned parameters, stimulus conditions 
could be calculated and assigned to an electrode to elicit a specific phosphene along 
the visual field. Fitting a camera and processing strategies that translated the per- 
ceived image into a phosphene map actually enabled the subject to identify simple 
patterns. Even though little is known about the long-term stability of phosphenes in 
such a crude prosthesis, these findings illustrate the importance of accurate phos- 
phene mapping strategies: Phosphene maps proved critical in translating the subjective 
percepts into discrete maps that could be used for clinical fitting of the visual 
prosthesis into a functional device. 
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19.4 Verbal Mapping 

Though mapping of phosphenes by retinal and optic nerve stimulation was reported 
long after the first cortically evoked phosphenes were characterized, the methods 
used were not very different. Several papers by Humayun et al. mention mapping 
of retinally evoked phosphenes [18-20], but they do not provide detailed informa- 
tion about mapping conditions, such as gaze-control of the subject or tactile refer- 
ences. In acute experiments Humayun et al. constructed absolute maps by asking 
the subject to verbally inform the experimenter of the quadrant (1-4), or clock hour 
(e.g., "9 o'clock") in which the phosphene was perceived [18]. As expected, the 
results confirmed that subjective phosphene location corresponded well with the 
electrically stimulated area on the retina. On the basis of these findings, they 
extended their observations to relative measures by providing simple patterns 
through multi-electrode probes on the retinal surface and asking the subject about 
their percepts in acute experiments [19]. 

Verbal mapping was also applied in a study using TMS to evoke phosphenes in 
sighted subjects [27]. Subjects were placed in a darkened room with eyes closed to 
facilitate phosphene percepts elicited by stimulation of the occipital lobes. Subjects 
reported if the phosphenes appeared in the upper or lower visual field, and whether 
the percepts were centrally or peripherally located. It appeared that peripheral pho- 
sphenes were encountered more often than central ones. Furthermore, phosphenes 
were more often observed in the lower-field than in the upper field. The first finding 
is unexpected, since the central visual field at the occipital pole (i.e. the foveal 
projection) is more accessible for TMS than the peripheral retinal projections 
located at more rostral aspects from the calcarine fissure. The authors speculated 
that TMS in this study may have activated peristriate cortical areas. 

Fernandez et al. [14] proposed an alternative method of phosphene imaging by 
verbal communication for sighted people. This method incorporates several training 
phases by using a clock-face division of visual space. Each of the 12 sectors is 
labeled accordingly and divided into annuli to produce an inner, middle, and outer 
portion, representing displacement between the fovea and visual periphery. In the 
training phase subjects are provided with a computer screen and learn to specify the 
projection of a light spot over the clock-face frame. Initially, spots of light are pre- 
sented onto a full outline of the frame and subjects are asked to indicate in which 
of the 36 sections the spot appears (hour and eccentricity). The second training 
phase is performed without sector labeling and subjects receive feedback about 
their performance. The last phase consists of phosphene localization without the 
frame and subjects again receive feedback on their performance. Each of the training 
phases are repeated until the subject achieves a required percentage correct perfor- 
mance level, before proceeding to the next phase. After training, testing consists of 
identification of the location of phosphenes in an imaginary frame. The authors 
predict that this method should be faster and should yield a higher spatial resolution 
and more discrete responses from the subjects than other absolute mapping methods. 
In addition, any effects of visuo-motor transformations required for drawing are 
eliminated by this method. 
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The authors mention that blind people could learn this method (though training 
with a visible frame would be impossible) with the help of a dartboard divided into 
12 sectors and three annuli, much like the method of Mladejovsky et al. [26] dis- 
cussed above. 



19.5 Mapping Studies Using Subject Drawings 

After their first studies on phosphene mapping using acute retinal stimulation 
(Sect. 19.3), Humayun et al. chronically implanted a retinitis pigmentosa patient 
without functional vision with a 16-electrode retinal implant [20]. They mapped the 
phosphenes by letting the subject draw the percepts on a drawing board positioned 
on the subject's lap. Similar to their preceding studies using verbal information 
discussed above, the constructed phosphene maps indicated that percepts corre- 
spond well with the retinal layout; i.e., electrodes temporally located evoked 
nasally perceived phosphenes (and vice versa), and superiorly located electrodes 
evoke inferiorly perceived phosphenes (and vice versa). Resolution appeared to be 
1.5° of visual angle. 

In another study a very similar drawing method was used to map phosphenes of 
sighted subjects with intractable epilepsy who were chronically implanted with 
subdural electrodes in the extrastriate visual cortex [21]. Subjects were asked to 
look at the center of a white board positioned 2 m away. The white board was 
divided into sections by horizontal and vertical median lines. The subjects were 
instructed to make drawings on a white paper, regarding the outline and location of 
the phosphenes. The paper was one tenth the size of the white board and had similar 
dividing lines. Phosphene shape, color and motion were also recorded. These draw- 
ings were then used to extract polar angle and eccentricity of the phosphene for 
mapping purposes. The results of this study showed that retinotopic maps could 
also be found on the lateral occipital cortex. 

Several TMS studies made use of drawings of phosphenes made by subjects, 
starting with an early study by Marg and Rudiak using normally sighted people 
[24]. For optimal phosphene perception, subjects were seated in a darkened room 
with their eyes closed. Subjects made drawings of the phosphenes and reported on 
characteristics such as the shape, color, brightness/vividness and position and dis- 
tance in the visual field relative to the fixation point. Besides detailed morphology 
of TMS -evoked phosphenes, the authors reported that phosphenes in the peripheral 
field of vision occur more frequently than central ones, in agreement with the findings 
ofRayetal. [27]. 

Subject drawings of perceived phosphenes were also applied in a TMS study 
with sighted subjects and two visually impaired subjects. One of the visually 
impaired subjects lost all functional vision at the age of 53 (subject was 61 years 
of age at the time of testing), while the other had a partial vision loss due to 
severe damage to the left striate cortex at the age of 8 (subject was in his early 
40s when testing took place) [5]. Phosphenes were mapped by letting the subjects 
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draw their percepts on a tilted drawing board 57 cm in front of them, while fixing 
their gaze on the center of the board. In a later phase of the study the subject was 
seated 50-200 cm from a white wall and was asked to indicate with a laser 
pointer to the location of the phosphene relative to a reference point on the wall. 
Subjects also traced the outline of the phosphenes with the laser pointer. The 
experimenter redrew the phosphene with a pencil. Interestingly, while retinotopy 
was clearly present in the sighted subjects and the subject with partial vision loss, 
the retinally blind subject did not show clear retinotopy and had a degraded spatial 
representation. In contrast, Brindley and Lewin [3] found a clear retinotopy in 
their blinded patient when using subdural electrodes. The authors speculate that 
the diffuse cortical excitation inherent in TMS makes precise stimulation impos- 
sible when cortical organization is interrupted due to total vision loss [5]. 

In a similar TMS study the authors made use of a digitizing tablet connected to 
a personal computer to directly convert the drawings to digital data [14]. After each 
TMS pulse the subject drew the image on the tablet, which was provided with a 
central pin for tactile reference to the center of the visual field. This phosphene 
mapping method showed that TMS was capable of evoking phosphenes in 17 out 
of 18 sighted people, and that phosphenes could be evoked along the entire visual 
field by stimulating the occipital cortex with single pulses. Blinded subjects often 
needed TMS pulse trains instead of single pulses, and in only 54% of these subjects 
phosphenes could be evoked. TMS is generally used to disrupt cortical function and 
although many phosphenes appeared as dots of light, spots of darkness ("scotomas") 
were also reported. 



19.6 Recent Simulation Studies Using Phosphene Mapping 

Simulation studies on the functionality and capabilities of visual prostheses are 
becoming more important (Chap. 16). Regarding phosphene mapping, simulation 
studies can be used to carefully control the test environment and allow a comparison 
of different mapping strategies. 



19.6.1 Tactile Simulations at Shanghai Jiao Tong University 

Ren and associates published two papers about novel ways to construct absolute 
phosphene maps based on simulation studies. Both studies used normally-sighted 
subjects who were presented with simulated phosphenes using a head-mounted 
display (HMD). The first study [4] made use of a touch screen (39 cm in width, 
30 cm in height) that was placed at eye level. The screen was provided with a tactile 
reference point in the center of the screen. Subjects were seated in the dark and 
fixated at the center of the screen by means of a chin rest. Their left index finger 
was placed on the reference point on the touch screen for tactile feedback, while the 
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right index finger was used to point at the simulated phosphene on the touch screen, 
much like the pointing hemisphere deployed some 40 years earlier by Brindley and 
Lewin [3]. The experiment compared two test conditions by presenting phosphenes 
with and without a reference grid projected in the HMD, which divided the visual 
field in 6 x 8 cells. Under both conditions, the experiment was preceded by a training 
phase in which the subjects could see their own response on the touch screen. In the 
first phase of training, phosphenes were presented in a predictable way, allowing 
the subjects to familiarize themselves with the equipment. In the second training 
phase, phosphenes were randomly presented. After that, the actual test was per- 
formed during which subjects could not see their response. Phosphenes were 
presented at 3°, 11° and 15° eccentricity [4]. 

The investigated parameters included dispersion of the responses (standard error 
of the response in mm), accuracy (the distance between phosphene and mean 
response in mm) and response time. Their results showed that in the presence of the 
reference grid dispersion, accuracy and response time tended to be lowest. In addi- 
tion, dispersion and response times increased when phosphenes were presented at 
larger eccentricity. The authors also showed that dispersion was larger in the left 
two quadrants of the visual field compared to the right two quadrants. They attrib- 
uted this result to the fact that the left hand was always used for tactile reference 
which interfered with pointing to a phosphene in the left half of the visual field. 

In a follow-up study, Ren and colleagues [31] used a very similar setup, but 
adapted their method to improve tactile feedback to the subject by overlaying a 19 
touch screen monitor with a 31x31 push-button array (41 cm in width, 35 cm in 
height). Tactile references were improved by (a) an elevated center button repre- 
senting the origin and (b) slightly elevated buttons along the horizontal, vertical and 
diagonals of the push-button array. Subjects could use both hands to localize the 
phosphene on the push-button array. In contrast to their earlier method, the screen 
with the push-button array was placed horizontally on a table in front of the subject. 
Training consisted of three phases. The first training phase permitted the subjects 
to familiarize themselves with the array by letting them feel the origin and the 
elevated buttons indicating the dividing lines. The second training phase provided 
the subjects with phosphenes localized in a restricted portion of the visual field. The 
third and final training phase provided the subject with 24 random phosphenes and 
the subject could observe the response in the HMD. The test phase consisted of 98 
randomly generated phosphenes. Again, dispersion, accuracy and response time 
were recorded. 

Compared with the unaided touch screen method dispersion was lower and 
accuracy more constant when using the push-button array. Furthermore, the sys- 
tematic left hemifield error observed in the first study was absent. However, 
response times were much higher (25 vs. 3 s in the earlier study) and the authors 
speculate that subjects spent most of this additional time on finding the tactile 
references and appropriate button on the array. Another possibility is the fact that 
the push-button array was placed horizontally in front of the subject, instead of 
vertically at eye level as in their first study. This setup likely demanded more of the 
subjects, since they had to translate visual field coordinates to a horizontal surface. 
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Though testing times may become long, the push-button array may prove a valuable 
method for testing subjects with little or no residual vision, because of the tactile 
references, better resolution and reduced localization errors. 



19.6.2 Simulations in Our Laboratory 

Dagnelie and Vogelstein developed and compared three different phosphene mapping 
methods based on phosphene localization simulation studies in four normally- sighted 
volunteers [6, 7]. An HMD with a 40° x 50° binocular display with 5 arcmin resolution 
was used to present phosphenes (see Chap. 16). The HMD provided subjects with a 
central fixation point and was equipped with a pupil tracker to monitor eye movement. 
Trials were aborted if the gaze deviated by more than 0.5°. All three methods were 
designed to mimic a prosthesis positioned over the primary visual cortex of one cere- 
bral hemisphere by presenting 32 randomly located phosphenes binocularly in one 
visual hemifield at eccentricities up to 20°. Phosphenes were round dots with a diam- 
eter of 20 arcmin at the fovea and increasing in diameter to 40 arcmin near 20° eccen- 
tricity in order to mimic cortical magnification. For each subject, one set of phosphenes 
was generated for use in all three tests to facilitate comparison between methods within 
a subject. Additional random sets were used in the touch screen and eye movement 
procedures, discussed below. Subjects came in for four or five 1-h sessions. 

The first method was much like the method of Chai et al. discussed earlier, 
deploying a touch screen (18"xl2", height x width). Figure 19.1 shows a subject 
performing this procedure. Subjects sat in front of the screen and held their left 
index finger on a tactile marker located halfway down the left edge of the screen. 
Subjects were told to place their right index finger immediately beside the left index 
finger on the screen and to align their fixation point in the HMD with this (unseen) 
finger as best they could. A phosphene was then presented, accompanied by a tone 
and subjects had to slide their right index finger across the screen to the location of 
the phosphene and lift their finger off the screen. A second tone signaled that the 
computer had registered the lift-off coordinates. This process was repeated until all 
32 phosphene positions were mapped; this process was repeated, for a total of three 
estimates per position. All data were obtained in a single session. 

The second approach recorded a saccade to the remembered phosphene location, 
using a calibrated pupil tracker in the HMD. The subject fixated on the central dot 
displayed in the HMD. After a warning tone, a phosphene momentarily appeared 
(400 ms) after which a saccade was made to the former phosphene location. The 
subject was required to briefly maintain gaze while the pupil-tracking software 
recorded the coordinates of the final eye position. This procedure was repeated for 
all 32 positions and repeated three times in a single session. 

In contrast to the absolute phosphene coordinates estimated with the first two 
techniques, the third method constructed a relative phosphene map. This so-called 
triadic distance comparison method compares distances among point triads, with 
map reconstruction through multidimensional scaling (MDS). During the test, subjects 
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Fig. 19.1 Subject performing the touch screen task employed by Dagnelie and co-workers. The 
left index finger is placed on a tactile marker representing the center of fixation, placed midway 
along the left side of the touch screen. The head-mounted display shows an internal fixation point, 
and the subject attempts to keep the line of sight in the HMD aligned with the tactile fixation 
marker. A pupil tracking camera inside the HMD is used to monitor steady central fixation while 
a phosphene is present. The subject's right index finger has just completed tracing towards the 
perceived phosphene location and is briefly held steady before being taken off the screen, marking 
the position. The scene camera, visible on the front of the HMD, was not used in this test 



again maintained fixation on the central dot on the HMD screen. Three phosphenes 
appeared sequentially and remained visible for 500 ms. Subjects numbered the 
phosphenes according to their appearance and reported which two dots were closest 
and which two were farthest apart. The experimenter keyed in the reply and started 
a new trial. Testing all possible triads including the 32 phosphene locations would 
have required 4,960 trials. To reduce this number, the 32 phosphenes were divided 
into four overlapping groups of 16 from which pseudo-random triads were pre- 
sented. Each pair in a group of 16 was presented four times, rather than the maxi- 
mum of 14 times. In this way, 160 triadic comparisons per group were made, for a 
total of 640 trials to complete all four groups; maps from the groups of 16 could be 
combined by virtue of the eight common points among "adjacent" groups of 16. 
Performing the 640 comparisons required two or three sessions. 

The MDS procedure consisted of building a similarity matrix for all pairs, in 
which ternary values were assigned for each response ("2" for closest, "1" for 
intermediate and "0" for the farthest pair), similar to the method from [26] described 
in Sect. 19.3. A Kruskal MDS procedure was then performed to reconstruct the 
two-dimensional dot distribution (mean = 0, SD=1) [23]. Note that the resulting 
map not only needs to be translated and scaled to allow comparison of the recon- 
structed coordinates with those obtained by the touch screen and eye movement 
methods but, due to the relative nature of triadic comparisons, it will also require 
rotation, and possibly a mirror imaging operation. 
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Figure 19.2 shows results of a complete set of tests for one representative 
subject. Touch screen results are shown in panel a, eye movement results in panel 
b, triadic comparison results in panel c, and combined results in panel d. The 
square symbols and connecting (dashed) lines in each figure represent the stimulus 
locations and the order in which they were presented; the connecting lines are 
presented only to facilitate comparison with the corresponding mean responses 
(drawn black lines; no data points, since each break point in the line is the center 
of gravity of the responses for that stimulus in multiple trials). Panels a and b show 
raw responses, while those in panel c have been transformed to optimally fit the 
stimulus coordinates, as explained above. 

A comparison between the three methods suggests that the touch screen and eye 
movement tests had better relative accuracy with increasing eccentricity, but overall 
had poor reproducibility (up to 25% test-retest variability). Furthermore, as judged 
by comparing the line sets in each panel, the touch screen data show a relative 
expansion along the horizontal axis, and a downward trend, while the eye move- 
ment test resulted in a horizontal compression; the break points in the triadic com- 
parison reconstruction map appear much closer to the phosphene coordinates, but 
this may in part be due to the optimized scaling. For the combination and compari- 
son in panel d we have therefore optimized the fits of the touch screen and eye 
movement data through translation and isotropic expansion. Averaging all three 
methods in panel d yields a map in which the lines representing the "grand mean" 
response bears a good, albeit somewhat distorted, resemblance to the lines connecting 
the phosphene coordinates. 

While these results are those for a single subject, the findings are typical of what 
was found in a half dozen others: touch screen responses overestimate horizontal 
eccentricity, while eye movement responses underestimate them. Triadic distance 
comparison test performance was better than the other two tests. This does not 
mean, however, that one can rely on this test alone: one or more absolute mapping 
methods are required to obtain an overall map of phosphene locations. 

Also, given the time-consuming nature of the triadic comparison tests, it may be 
best to concentrate efforts using that test on clusters of closely spaced phosphenes 
while using the absolute techniques to establish relationships between such clus- 
ters. By applying translation, rotation and scaling to achieve maximum correspon- 
dence among data from different tests, one can hope to attain the most accurate maps. 

One should bear in mind that in an actual prosthesis wearer there will be no 
stimulus map to which the results can be fitted. Nonetheless the results obtained 
in our lab inspire some confidence. Dagnelie and coworkers [7] computed a 
distortion metric from the distance estimation errors for all possible phosphene 
pairs across the three tests for all five subjects tested with a uniform phosphene 
set. Three subjects had distortion scores under 15% for all tests. Combining maps 
by averaging the data of all three tests reduced the errors below 10%, which 
should enable adequate image recognition. The authors conclude that the three 
procedures, especially in combination, permit the construction of distortion 
maps with sufficient fidelity to enable shape recognition by future prosthesis 
wearers. 
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Fig. 19.2 Results of the three phosphene mapping techniques used in our laboratory, and com- 
bined results, for one representative subject. Each panel shows the results for one test; phosphene 
locations are identical in all four panels. Connecting lines represent the order of presentation in 
the touch screen and eye movement tests, and are shown to allow a comparison between the 
stimuli and corresponding means of the response in multiple trials (a-c) or methods (d). 
Coordinates are in screen pixels in the HMD; each pixel subtends an angle of approximately 
5 arcmin. Results in (c) have been transformed to obtain optimal correspondence with the coor- 
dinates of the 32 phosphenes 
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19.7 Concluding Remarks on Phosphene Mapping Techniques 

Various absolute and relative mapping procedures were discussed in this chapter. 
Absolute mapping provides estimated phosphene coordinates, while relative mapping 
provides phosphene positions only with respect to each other. Due to eye movements, 
mapping by sequential electrode activation may still yield unreliable relative phos- 
phene coordinates, but absolute mapping is inherently subject to position errors if 
gaze is not monitored. Relative mapping of closely spaced phosphenes yields more 
reliable information about phosphene positions with respect to each other, which will 
be important when trying to present arbitrary shapes to a prosthesis wearer. 

We reviewed more than a dozen absolute mapping techniques using a variety of 
pointing, drawing, verbal, and eye movement methods. Advantages of most of 
these absolute mapping procedures are their technical simplicity and the short time 
required to obtain a phosphene map. Especially when performing acute experi- 
ments in the operating room with time and equipment restrictions, absolute mapping 
by verbal communication may be the most convenient method. Data can be digi- 
tized on the spot by a drawing tablet, or recorded by the experimenter in the form 
of crude coordinates. With chronic implant wearers or visually impaired subjects in 
a laboratory setting, detailed information can be obtained using a touch screen or a 
dart board or clock face with tactile markings. Tactile markers and training improve 
accuracy. Drawings can be advantageous when phosphene shape is of interest. 

Disadvantages of these techniques are their inaccuracy and the difficulty resolving 
phosphenes located closely together, especially by subjects with long-standing 
vision loss. Visuo-motor translation may affect the results, especially when phos- 
phene location or shape is indicated by drawing. Disadvantages of verbal descrip- 
tions, paper drawings and pointing to a surface other than a touch screen include the 
need to re-draw the data in a visual field map, or to digitize them into a computer. 
Finally, some of these methods can only be successfully employed by individuals 
with functional residual vision (e.g., using a laser pointer). 

Relative mapping methods require subjects to provide details about the relation- 
ship between different phosphenes. The techniques we reviewed varied in phos- 
phene presentation, using timing or other attributes to distinguish two or more 
phosphenes, but also in response modalities and analysis methods. All these techniques 
tended to be more complex and time consuming than the absolute techniques. This 
may not be a serious problem in subjects with long-term implants, as the benefits 
of careful mapping in increased ability to convey visual information to the prosthe- 
sis wearer will far outweigh the cost in time. 

Finally we learned that a combination of well-chosen absolute and relative mapping 
methods may yield accurate maps with acceptably small distortions. There is still a 
need to further elaborate some of the techniques beyond what has been described in 
the literature thus far, but many of the elements for reliable and efficient phosphene 
mapping procedures appear to be available. The principal remaining task is to perform 
comparisons of promising techniques, and choose optimal combinations. 
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Chapter 20 

Prosthetic Vision Assessment 

Marilyn E. Schneck and Gislin Dagnelie 



Abstract As visual prostheses continue to evolve, assessing their efficacy assumes 
paramount importance. This chapter identifies some of the key questions and issues 
that arise when planning and designing such assessments, in order to help point the 
way forward. 

High quality evaluations will naturally follow basic scientific principles such as 
including pre-operative as well as post-operative testing. Evaluations should 
include both visual function and visual task performance. Improved visual function 
tests may need to be developed or adapted that are suitable for the levels of vision 
afforded by current and near-term prosthetics. In assessing task performance, the 
choice of tasks to be assessed is critical, and can greatly influence the results. 

Longer-term follow-up testing after periods of acclimatization and training are 
also necessary, with control groups receiving alternative training such as more con- 
ventional rehabilitation or interventions. 

Self-assessment of difficulty in performing daily living tasks is also important, 
as are the more subjective assessments of user satisfaction. 

As the technologies continue to evolve, there will be a changing dynamic involving 
the steadily improving capabilities of the technology and the unique needs of a 
growing number and more diverse target population. 
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ADL Activities of daily living 
ALS Activities of life satisfaction 
BaLM Basic light and movement test 
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FrACT Freiburg acuity test 

HR-QOL Health-related quality of life 

IADL Instrumental activities of daily living 

O&M Orientation and mobility 

VFQ Vision function questionnaire 

VEP Visually evoked cortical potentials 



20.1 Introduction 

In order to assess the effectiveness of prosthetic vision in context, we must ask the 
question "Effectiveness for what"? Answering this question requires an under- 
standing of the realistic goals for this new technology. The prospect of "restoring 
vision to the blind" (e.g., [40]) has, of course, been received with great enthusiasm. 
However, the richness of our visual experience belies the complexity of the neural 
system delivering it, thus making it unrealistic for a prosthetic device to provide 
vision in its full and richly complex form. For instance, the number of electrodes in 
current devices (16-1,500) is many, many orders of magnitude fewer than required 
to carry the wealth of the information from the 1 20 million photoreceptors in each 
eye along over a million fibers of each optic nerve to the 140 million highly orga- 
nized neurons in each hemisphere of primary visual cortex, which in turn send them 
on to the many other regions of the cortex devoted to specific aspects of visual 
processing. 

The multitude of neurons form a complex network with feed-forward, lateral 
and feed-back signaling giving the visual system its complex imaging power. 
Subsequent to loss of photoreceptors in outer retinal diseases such as retinitis pig- 
mentosa, there are significant losses in both the inner nuclear layer (e.g., bipolar 
cells) and ganglion cells [50, 81, 91]. Furthermore, there is significant remodeling 
of neural retina and thus local neural networks following photoreceptor loss 
[64, 65]. Thus, whether implanted in the retina or the cortex, the implants will not 
have the full analytic power of the intact visual system. Currently, prostheses devel- 
opers are working to increase the number of electrodes and their density to improve 
resolution (e.g., 1,024 by Troyk's group [98]). Performance has been shown to 
improve with increasing number of electrodes [30, 42]. ' However, no matter how 
many electrodes in use, multiple phosphenes (discrete sensations of light), of vari- 
ous sizes shapes and hues, depending on the state of the post-receptoral retina and 
visual system, will be generated. How will these phosphenes be used to represent 
the environment? Past visual experience will certainly influence interpretation, but 
the recipient will have to undergo prolonged training to learn to use these signals. 



1 The success of cochlear implants is often cited as a hopeful indication of what can be accom- 
plished by a sensory prosthesis. Only six electrodes stimulating the cells of the auditory nerve 
enable the wearer to understand speech at near-normal levels. If one assumes the same ratio of 
electrodes to nerves, hundreds of electrodes are projected to be required [108]. 
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Thus, prosthesis recipients may have to make sense of a "blooming, buzzing confusion" 
that William James (1842-1910) said faced a newborn. 

Bearing these considerations in mind, it becomes clear that "restoring vision to 
the blind", is an amorphous goal that is unreachable in the foreseeable future. In fact, 
the goal of prostheses is not to recreate normal vision, but to provide visual percep- 
tion that however limited in scope is useful to the individual [69, 70, 78, 108]. 

How can we determine whether prosthesis aided or provided vision is useful to 
the individual? This requires well-defined, measurable goals for evaluation of prog- 
ress and demonstration of successful outcomes. Widely accepted outcome mea- 
sures and means of measuring them, and criteria for "success" have not been 
developed for prosthesis implantation. Standards have not been set. The fact that 
clinical trials are already in progress (e.g., SecondSight, Intelligent Medical 
Implants are conducting Phase 2 Clinical Trial) and others are planned makes more 
pressing the establishment of relevant outcome measures as criteria for success. 
Once such outcome measures are selected and developed, the means to assess per- 
formance with respect to these outcomes are specified, and testing has been carried 
out, progress can be assessed. The lack of well-specified relevant outcomes and 
means to assess them is a major hurdle in the future of prostheses. 2 

For most of us, activities of daily life rely on vision. Visual performance (actions 
incorporating and guided by visual input) is a very complex phenomenon involving 
other senses, motor skills, memory, prior knowledge, feedback, experience, prac- 
tice, etc. The basic building blocks for vision performance are sensory visual func- 
tions such as motion, color, luminance, contrast, and orientation. These contribute 
to object identification and localization. These in turn feed into higher order visual 
areas that integrate the visual information with other senses, motor systems, cogni- 
tive systems, and memory the ensemble of which guides the actions that form the 
tasks of daily living. In assessing prosthetic vision, we should ideally find ways of 
measuring these functions and task performance. 

For simplicity, we restrict the discussion that follows to consideration of indi- 
viduals with particular characteristics. Most prosthesis recipients to date have lost 
vision as a result of retinitis pigmentosa (e.g., [21, 22, 30, 51, 105, 118]). Therefore, 
we assume a target population of individuals with long-standing retinitis pigmen- 
tosa (RP), a progressive disease of the photoreceptors. Nearly all RP patients have 
some residual vision, typically one or more small peripheral islands in an otherwise 
non-functional retina [22]. 3 Since these individuals have had some vision well into 



2 Lengthy discussions of these issues took place at a special interest group meeting at ARVO 2007 
(organized by author MES and contributed to by author GD) [82] and a symposium hosted by the 
Smith-Kettlewell Eye Research Institute in San Francisco (October 2007) [83]. These meetings 
highlighted the complexity of the problems and demonstrated that more work needs to be done 
before specific recommendations of protocols and tests will be established. Some of the content 
of this chapter has been gleaned from those meetings. 

3 The degree of residual vision function is highly light-level dependent in RP patients; individuals with 
retinal disease often require unusually high light levels to attain their best vision function [85, 92]. 
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adulthood, they have not learned many of the adaptive tools of long-term blind 
individuals. For example, they are not Braille readers, but may use a cane. 4 

Rather than attempt to prescribe specific tests procedures, the remainder of this 
chapter is intended to raise and discuss the various issues relevant to visual assess- 
ment in patients with prostheses. 



20.2 Principles for Assessment of Prosthetic Vision 
20.2.1 Experimental Design 

Studies of the effectiveness of an intervention or technology, such as clinical trials 
of prostheses will incorporate a repeated measure design in which the individual's 
performance at one time point (before the intervention) is compared to that at other 
time points (after the intervention). The comparison of pre- and post-intervention 
results (the difference) is used as an index of the effect of the intervention. One 
useful way to express results is to report whether individuals cross some important 
criterion level of vision for example from "visually impaired" to "normally sighted" 
following treatment. In many clinical trials, the intervention's success or failure is 
judged with respect to a criterion level of visual function, usually visual acuity. In 
the context of prosthesis implantation, many "interventions" will occur (learning, 
prosthesis implantation, training and rehabilitation). To assess the effect of each 
intervention requires careful and broad assessment before and after its occurrence 
using scientific principles. 

An appropriate control group is an essential element in clinical testing. Control 
groups are comprised of individuals who differ from the treatment group, to the 
extent that is possible, only in that they do not receive the intervention. Comparison 
of the test results of the two groups is an important component of assessing the 
effectiveness of the intervention. 



4 Another set of issues and considerations arise in the case of long-term blind individuals who have 
adapted an array of methods that enable them to accomplish most tasks. For simplicity, we do not 
consider this population, though acknowledge that prostheses may be relevant to this population 
in the future. There are reports of recovery of vision following prolonged blindness (e.g., [35, 41, 
74]). In some cases, sensory vision recovered fairly quickly, though learning to recognize objects 
and people and to use vision to guide daily activities took considerable time and effort. One such 
patient never became comfortable relying on vision, preferring instead to close his eyes in tough 
situations [41]. This indicates that in some conditions leading to blindness, the visual system 
remains intact, but that things are not immediately recognized by sight; images must become 
associated with the information from other senses used before the onset of sight. 



20 Prosthetic Vision Assessment 389 

20.2.2 The Importance of Pre-operative Testing 

Thorough, scientifically valid pre-implant sensory vision assessment is essential for 
many reasons. Pre-treatment visual function is typically an inclusion/exclusion 
criterion for participation, though too rarely are these assessments well conducted 
or described. Prosthesis developers have suggested pre-operative assessment to 
select implant recipients [117]. Psychophysical and electrophysiological testing has 
shown that poorer residual vision is associated with reduced sensitivity to stimula- 
tion [117]. In order for prostheses to effectively provide useful vision, the visual 
apparatuses proximal to the prosthesis (e.g., all post-receptoral elements from eye 
to visual cortex for sub-retinal implants) must retain functionality, enabling them to 
receive and transmit the prostheses signal. In cases of retinal disease, such assess- 
ment may be accomplished by eliciting phosphenes by electrical stimulation or 
pressure to the eye, for example to determine whether this elicits a percept. In the 
case of post-orbital prostheses, the solution is more complex, and may involve 
magnetic stimulation, for example. 

As described in the context of experimental design, only by careful measure- 
ment of pre-operative vision can the benefits (or losses) of the implant be deter- 
mined. This is particularly important because many or most patients/potential 
recipients will have some rudimentary residual vision. For example, patients with 
retinitis pigmentosa (RP) form a large portion of the prosthetic implant candidates 
having undergone and currently undergoing clinical trials [21, 30, 51, 60, 105]. 
Only very rarely does RP result in total blindness. As mentioned earlier pre- 
operative testing will allow researchers to determine the degree and characteristics 
of the residual vision. This is essential for judging whether and to what degree the 
prosthesis improves vision. 

At least as important, but considerably more difficult, is the pre-operative evalu- 
ation of the vision-related skills and abilities of the individual. The range of tasks 
we carry out each day is enormous. Which are most "important" and should be 
assessed? There is no generally accepted answer. Individual recipients will have 
different priorities. Skills often assessed in low vision population include reading, 
face recognition, orientation and mobility and other simple activities of daily living 
(ADL; e.g., eating, dressing, washing) and more complex instrumental activities of 
daily living (IADL; more complex ADLs such as shopping, cooking). Creating 
appropriate tests with quantifiable measures for ADLs and IADLS poses consider- 
able challenges. This issue is discussed in a later section. 

Finally, assessment of the degree of difficulty an individual experiences when 
performing activities (ADL, social, recreational and vocational activities) is of 
value. A number of questionnaires have been developed and validated for assess- 
ment of low vision patients, though none exist for this population of individuals 
with very little residual vision who become prosthesis recipients. This issue is dis- 
cussed in a later section. 

Discussion of pre-operative (and post-operative) assessment strategies have been 
also been discussed elsewhere [25]. 
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20.2.3 Post-operative Assessment 

The reasons for post-operative assessment are self-evident. Post-operative evaluation 
will include all of the aspects of vision assessed pre-operatively. Initially, however, 
the emphasis will be on establishing that the prosthesis is functional and stable 
(delivers phosphenes in a repeatable manner). Post-operative assessment of the 
prosthesis recipient will occur over an extended time period, beginning soon after 
surgery and undoubtedly lasting for years. During this time both learning to use the 
prosthesis and evolution of the prosthesis will occur. 

Learning how to best interpret and use the signals from the prosthesis will be an 
ongoing process for the recipient (e.g., [25, 51]), most likely carried out interac- 
tively with the prosthesis team. Dobelle (2000) reported that a long-time wearer of 
a cortical implant initially was unable to recognize letters, underwent long-term 
training (10 days) and after prolonged continued practice had acuity of 20/1,200 
[31]. Learning how to interpret signals in order to form basic visual images will 
likely precede that of more complex visual perceptions and the use of the prosthe- 
sis-driven vision to perform tasks. The learning curve for prosthesis use is not 
known, and will certainly vary among individuals. Meanwhile prosthesis develop- 
ment will be continuously carried out, leading to alterations of the signal processing 
and stimulation routines. Measurement time points need to be set and all recipients 
should be seen at each time point using a pre-determined, set protocol, if possible. 

Hippocrates put forth the important "First do no Harm" principle of medical 
intervention. The possibility exists that in some cases prosthesis implantation may 
negatively affect the recipient's vision. Damaging surrounding functional tissue 
may reduce or destroy any residual vision. Kiser et al. reported that at least one 
prosthesis recipient's vision deteriorated due to surgical complications (and three 
developed cataract leading to a loss of vision) [53]. The use of the prosthesis may 
reduce visual performance in other ways. Any visual signal, even non-structured, 
"noise" resulting from the prosthesis may interfere with learned strategies by drawing 
attention away from reliable information from other senses (e.g., tactile and audi- 
tory feedback from a cane). Such losses can only be documented by comparison to 
pre-operative testing of visual function and performance. One may switch the 
device on and off to determine its utility and or its interference with performance, 
but not its potential effect on anatomy. 



20.2.4 Methodological Issues in Pre- and Post-operative 
Vision Assessment 

20.2.4.1 Potential Approaches 

Two approaches offer themselves for assessing vision: psychophysics and electro- 
physiology (recording of electrical responses from the visual system). 



20 Prosthetic Vision Assessment 391 

Psychophysics is a discipline that determines the relationship between dimensions 
of a physical stimulus and perception. Psychophysical techniques can be used to 
measure threshold (minimal detectable or minimally discriminable) and supra- 
threshold vision. Virtually all standard clinical measures of vision are psychophy 
sical threshold measurements (for example, visual acuity, contrast sensitivity, visual 
fields, color discrimination). Supra-threshold measures are more often made in the 
laboratory and include for example, contrast matching, figure-ground segregation, 
brightness matching, visual search etc. Psychophysics is an elaborate, well-defined 
discipline with its own rules of operation. Psychophysical procedures are designed 
to minimize observer and tester bias (see e.g., [23]). 

Visual electrophysiology involves the recording of electrical signals (voltage 
changes) generated in response to the stimuli of interest. The visually evoked (cortical) 
response (VEP), is the most commonly used method for assessing basic visual 
function. The VEP can be used to assess both threshold and supra-threshold vision. 
It is recorded by placing electrodes on the scalp over the primary visual cortex (at 
the back of the head) and other associated visual areas of interest. In recent years, 
increased signal to noise ratio (SNR), new methods of signal processing and data 
extraction have greatly extended the range of visual functions that can be measured 
and the sensitivity of these tools. Currently sweep VEP measures [103] are used to 
determine visual acuity, contrast thresholds and Vernier thresholds in clinical set- 
tings (e.g., [1, 73, 107]). Many other characteristics of vision, including higher 
order visual processes, can also be measured by the VEP. The VEP is an objective 
(bias-free) measure. 

Though electrophysiology (and other imaging techniques not described) cer- 
tainly have a role in prosthesis assessment (particularly the VEP in pre-operative 
and early post-operative settings for non-cortical implants), psychophysical testing 
will undoubtedly dominate vision assessment. VEPs measure activity of the primary 
visual cortex. It is possible that VEPs are recordable, but that the individual has no 
access to this information, i.e., cannot use it to see. 

If the fellow eye retains some vision, then testing of the prosthesis is most easily 
carried out with the fellow eye patched (i.e., monocularly). However, assessment of 
the individual's vision and functional ability should always be made binocularly, as 
that is the way one goes about his/her business. 



20.2.4.2 Avoidance of Bias 

Two sources of unwanted bias may affect experimental results. Experimenter bias 
occurs when a researcher unconsciously manipulates procedures to achieve an 
expected or desired outcome, potentially skewing the results. Patient bias arises 
from either a "placebo effect" (effect based on the power of suggestion and/or 
induced by having been seen by a trusted expert and receiving a "treatment"), or 
the desire to please the investigator. 

To avoid these biases a double blind approach is often used for testing. In this 
case, neither the examiner nor the patient is aware of whether the subject has 
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received the treatment/intervention or not. To assure that the masking is effective, 
placebo treatments and/or sham procedures that closely resembles the intervention 
may be performed. Comparison of the sham or control subjects' and patients' per- 
formance reveals the true (unbiased) treatment effect. 

Change over time in the sham and treated groups may also be compared. If the 
change across time in the sham group is as large as and in the same direction as that 
of the treatment group, it is concluded that the treatment (implant) has not affected 
performance. 

Appropriate psychophysical techniques for vision measures within the double 
blind (or masked) protocol (in which neither the experimenter nor subject is aware 
of the subject's group (treatment vs. control)) further reduce both sources of bias. 
Methods for development, administration and analysis of questionnaires using a 
masked protocol reduce bias for these instruments 



20.2.4.3 Criteria for Sound Testing 

To be scientifically sound and produce credible results, test procedures must be 
carried out in such a way as to (1) specify stimulus characteristics such as size, 
intensity, spectral characteristics, and duration in sufficient detail to be replicated; 
(2) use an objective, criterion free response (such as offered by a multiple alterna- 
tive forced choice (see below)); (3) assure that the outcome cannot be attributable 
to extraneous (non- visual) cues (including bias); (4) allow specification of the like- 
lihood that the reported outcome occurred by chance; (5) have predefined methods 
of analysis and predefined criteria for success/failure. These principles apply to 
both sensory vision and vision performance testing. We address these principles 
more fully below. 



20.2.4.4 Forced Choice Procedures 

Forced choice psychophysical procedures minimize observer and tester bias. As the 
name implies, in this procedure the stimulus is present in one of a number of spatial 
or temporal intervals, and the observer's task is to select the correct interval. Forced 
choice procedures are equally appropriate and simple to use with supra-threshold 
stimuli. In this context, the observer's task is to choose from among several stimuli 
to pick the one with the attribute of interest (brightest, highest contrast, fastest, etc). 
Insertion of "blank" or "catch" trials further allows the experimenter to assess and 
remove biases. To determine whether the observer did in fact detect/identify the 
target correctly, performance is corrected for guessing (expected percent of the trials 
the patient would get correct by guessing) as follows: 

(observed percent correct - expected percent correct by chance) 

True percent correct = . 

(1 - expected percent correct by chance) 
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There is a lower limit on the number of trials necessary for assessing whether 
performance is beyond chance. In effect, increasing the number of alternatives on a 
trial increases the information from that trial and reduces the number of trials required 
to assure that the observer's performance is not based on chance. Minimizing the 
number of trials per test is crucial, particularly when many measures are to be made. 
It is thus important to use an efficient rule for varying the parameter of interest to 
arrive at "threshold" with the minimum number of stimulus presentations (and mini- 
mize errors based on assumptions of the slopes of the psychometric functions). There 
are many in use (e.g., Pest, bestPest, Quest, Psi, adaptive staircase, Method of 
Adjustment, the Method of Limits, and Method of Constant Stimuli) [23]. Threshold 
is the minimum level of some stimulus parameter that can be detected, resolved or 
discriminated reliably. In practice, threshold is typically defined as the stimulus level 
that produces a criterion percent correct after correction for guessing. 

To more thoroughly consider response biases, and make better use of the infor- 
mation to be gained from each trial one may analyze data within the context of 
signal detection theory [39, 59]. 



20.2.4.5 Response Time 

In psychophysical testing, the time between stimulus presentation and the response 
(often called "reaction time") provides additional important information. It is an 
indication of the individual's confidence in his/her response and thus the difficulty 
of the task. For example, responses are faster to stimuli that are well above thresh- 
old than to near-threshold stimuli. In the present context, response time has other 
significance. To be useful in the "real world", performance must be at must be at 
least as efficient as that obtainable by other (non-visual) methods. An individual 
may be able to locate a banana on a table using prosthetic vision after several minutes 
of searching, but be able to accomplish the task by searching using his/her hands in 
seconds. In other situations, such as crossing the street, fast responses are more 
critical. If an extended time is required for image interpretation and arrival at 
correct decisions, the information may be of limited value. 

A criterion response time for passing/failing can be developed for each measure 
based on (multiples of) the response time of normally sighted observers, the time 
required for a blind individual to achieve the task using other strategies, or with respect 
to the window over which the response would be useful. Another important consider- 
ation is the degree to which the individual desires independence. An individual who 
does not want to be helped may tolerate much longer performance times. 



20.2.4.6 Task (Perceptual) Learning 

An individual's performance on a task improves with repeated performance of the 
task and finally asymptotes at peak performance when learning is "complete" (i.e., 
asymptotic performance along the dimension of interest is reached). At each measurement 
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time point (both pre- and post-operatively), it is crucial that testing continue until 
this asymptote is reached. This is important for pre-operative testing because we 
cannot otherwise distinguish whether any observed benefit at a later time point is 
due simply to continued learning or from the advantage conferred by the implanted 
device. Conversely, we may underestimate benefit by not measuring maximum 
performance at the later time point. Repeated testing of a task after task perfor- 
mance has reached its asymptote also provides an estimate of test-re-test repeat- 
ability. The 95% confidence limit of this variability sets the criterion for true 
change imparted by the implant (and subsequent interventions including training 
and rehabilitation). This is crucial for establishing whether any change is statisti- 
cally significant at a criterion probability. 



20.2.4.7 Establishing Criteria for Meaningful Change 

After defining significance in the statistical sense, there remains the question as to 
whether the measured benefit or loss has any clinical or practical significance. For 
visual acuity, this criterion is generally 0.3 log units [24], which is equivalent to a 
halving of the required target size, for example going from 20/2,000 to 20/1,000 or 
20/40 to 20/20. Individuals with acuity improvements of this magnitude also dem- 
onstrate large gains on the vision specific subscales of the VFQ-25 questionnaire 
[18]. However, these criteria are based on statistical considerations (test-re-test 
repeatability). A similar criterion (0.3 log units) is also useful for contrast sensitivity. 
Criteria of meaningful change have not been established for other aspects of visual 
function, or visual performance. In the case of visual performance (of tasks or 
activities of daily life), defining a criterion for meaningful improvement may be 
more difficult. 



20.2.4.8 Light Level 

There are standards for light level for acuity measures (80-120 cd/m 2 ) [14]. In normal- 
ly-sighted people, acuity declines with luminance below the standard values and is 
constant across higher light levels. In those with disease, however, both sensory 
vision and function are highly dependent on light level over a broader range (e.g., 
[85, 92]). Many affected individuals require higher-than-normal light levels to see and 
to function. For example, Kuyk et al. showed that reducing light level had an adverse 
effect on mobility in patients with age-related macular disease [54]. In RP patients, 
including those with Usher's syndrome, field size is critically dependent on light 
level, shrinking with diminishing light levels. This is evident in six patients with 
Usher's syndrome illustrated in Fig. 20. 1 (Haegerstrom-Portnoy, personal communi- 
cation). The field diameter of Subject TL increases by a factor of 10, from a mere 2.5 
degrees to 25 degrees, over the light range tested (1 to 1,000 cd/m 2 ) and would most 
likely continue to increase with further increases in luminance. Therefore, pre- and 
post-implant testing should be carried out over a range of light levels, including those 
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Fig. 20.1 Horizontal diameters of visual fields are plotted across luminance. Each symbol identifies 
an individual subject 

well above what is normally required, say at daylight levels (10,000+ cd/m 2 ) in order 
to maximize the contribution from the remaining functional retina beyond the 
prosthesis. This may be done in log unit steps of 10 cd/m 2 (white paper under living 
room level), 100 cd/m 2 (white paper under office light levels), 1,000 cd/m 2 (white 
paper outdoors in overcast) and 10,000 cd/m 2 (white paper outdoors in sun). 



20.3 Vision Assessment in Prosthesis Recipients: Overview 

We consider vision assessment as two components. They are, in our terminology, 
visual function and visual performance. Visual performance is itself comprised of 
two components: objective (measured) and subjective (self-reported) ability to 
accomplish tasks. In this regard, the point of view of this chapter is similar to the 
three-pronged approach suggested by Wilke et al. [112]. 



20.3.1 Visual Function Assessment: Overview 



Visual function forms the basis of evidence for evaluating benefit/risk to relevant 
parties (e.g., other researchers, funding agencies, the FDA). Acceptable outcome 
measures of clinical trials have classically been visual function, most notably visual 
acuity, as described above. Four aspects of vision accepted by the FDA are, with 
constraints, visual acuity, visual fields, contrast sensitivity, and color vision [24]. 
Visual function may be more reliable, objective, and sensitive than either aspect of 
visual performance; however, it may be the least relevant to real-life conditions to 
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be faced by the prosthetic recipient. Detailed description of prosthesis-generated 
visual function will guide future prostheses work. As noted by Chader et al. [20], 
"what is needed is an accurate and reproducible method to link visual testing with 
real-world functional capacity in individuals with very low vision". 



20.3.2 Visual Performance Assessment: Overview 

20.3.2.1 Measured Visual Performance 

Both sighted and blind individuals integrate input from a variety of sense modalities 
as well the motor system, and cognition to learn skills to accomplish tasks. The 
accomplishment of tasks through vision in combination with other senses and skills 
is the most complete, and, arguably the most important index of the value of the 
vision afforded prosthetic device. 

Numerous studies have shown only weak (though statistically significant) asso- 
ciations between visual function and task performance, including performance of 
ADL (e.g., [47, 54, 93]). This is not surprising since with training and practice (i.e., 
rehabilitation), most tasks can be accomplished without any vision. An individual's 
visual performance cannot be inferred from even an extensive array of well-selected 
sensory vision measures. In practice, a very limited number of such measures will 
be made. Thus, it is necessary to meet the challenge of developing tests of compo- 
nents of tasks of interest, or means of measuring directly performance of the actual 
tasks. Development of valid, relevant performance tasks is a hurdle that must be 
overcome to advance the field of visual prostheses. 

20.3.2.2 Self-Reported Visual Performance 

Self-report, using properly developed questionnaires and appropriate analysis tools, 
serves many functions. These instruments enable us to: assess extraneous variables 
that could affect intervention success (e.g., depression and other co-morbidities); 
assess a wider spectrum of functional and performance aspects of vision than are 
practical via direct measurement; assess the impact of interventions on global indices 
of well-being such as quality of life and self-perceived disability. Instruments also 
efficiently gather large amounts of information [61]. 



20.4 Visual Function Assessment 

Given that function is only loosely related to sensory vision (except in the extremes), 
the choice of visual attributes to be measured could be made on the basis of a number 
other criteria such as acceptability to relevant agencies as an outcome measure, 
value in guiding further prosthesis refinement, scientific "curiosity", or determining 
the individual's legal eligibility for benefits. 
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The selection and design of vision assessment tests must take into account the 
expected level of function but at the same time cover a wide level of function to be 
suitable for pre-implant and post-implant assessment. As noted, prosthetic vision in 
the near future is likely to be fairly crude, but it will exceed the visual function with 
which the subject presents. For preoperative assessment, tests should be designed 
for those with extremely limited vision. However, the prostheses may result in large 
improvements, so measures must be able to assess the higher function as well. 
There is no consensus on which aspects to measure, but the battery must be suffi- 
ciently brief to assure patient comfort. Tests that are inexpensive and easy to admin- 
ister are most likely to gain wide acceptance. 



20.4.1 Candidate Measures 

Light perception is the ability to tell whether one is in light or darkness, or more specifi- 
cally, whether the visual field is light or dark. This level of vision is well below the target 
for outcomes in implant trials. However, measurement of light perception, is invaluable 
in pre-operative testing, particularly for determining (1) whether the visual pathway 
proximal to the implant is functional and (2) and to define minimal light levels for 
further testing. Dagnelie [25] has suggested measuring light perception (detection) as a 
threshold task; at what level can the individual first detect light (threshold) and out to 
what range? The idea behind this test is that, considering the vast range of light levels 
over which the visually normal person can function, determining the operating range 
may have a greater ability to classify severe vision loss than almost any other measure. 
Knowledge whether an individual has light perception prior to implantation is necessary 
for comparison to post-implant visual function gain. However, vision at the level of light 
perception is of very limited value in terms of the recipient's daily function. 

Light projection (or localization) is the ability to indicate from which direction 
a light originates. Assuming clear ocular media, light projection in the eye follows 
the laws of geometric optics, so that the retinal location of the illuminated area 
(nasal vs. temporal, up vs. down) is predictable. However, in implant recipients who 
have no vision outside the areas driven by the implant, perception of location is 
likely to be driven by prosthesis location with respect to the fovea or fixation loca- 
tion, at least initially. The recipient will, presumably, learn to remap visual space 
based on the implant's location and function and to "fit" visual space into this area. 5 



5 A difficult aspect of any vision task using a target of limited spatial extent will be locating the target 
in visual space. This is most difficult for devices with which field of view does not follow eye move- 
ments, which are currently the most common. Individuals can learn to suppress eye movements in favor 
of head movements, but this is difficult and perhaps inadequate. Some prosthesis developers have 
addressed this problem by yoking the external camera or its image to eye movements or implanting the 
photo-detector/camera in the eye (e.g., [22, 37]). Though prostheses are placed to tap into foveal pro- 
cesses, the (retinal) prosthesis may be displaced from the fovea. Prosthesis wearers must unlearn the 
tendency to move the eyes to foveate. For prostheses in which the receiving and stimulating elements 
are co-located in the parafovea, the recipient may need eccentric viewing training, currently used for 
patients with age-related macular degeneration with absolute scotomas that involve the fovea [72]. 
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Light projection enables the individual to locate light sources (such as windows 
or doors) with respect to his or her position and can thus aid navigation. 

The visual field is the area of space within which an individual can detect the pres- 
ence of a visual stimulus. The visual field of a normal human eye measures (from the 
point of fixation) 100 degrees temporally, 60 nasally, 75 superiorly and 60 inferiorly 
[8]. Binocular (using both eyes) visual fields are approximately 200 degrees wide and 
135 degrees tall, with a region of binocular overlap that is 120 degrees wide. 

In visually normal observers, sensitivity varies considerably within these limits 
[97], so that the size of the measured visual field is strongly dependent on target 
size and luminance. This dependence is more dramatic in those with RP. Very small 
fields impair mobility. 

The integrity of the field is also an important measure. If a bigger or brighter 
target than normal is required for detection in a region, that region has a relative 
field loss or relative scotoma in that region. An absolute field loss, such as an abso- 
lute scotoma, is a region in which the patients cannot detect any target. 

Visual field testing (perimetry) merely require target detection, rather than local- 
ization and is carried out two basic ways: with stationary (static perimetry) or moving 
(kinetic perimetry) targets. In kinetic testing, targets are slowly brought from random 
locations outside the far peripheral field toward the point of fixation. In contrast, in 
static perimetry stationary targets appear briefly at any random location irrespective 
of distance from the fovea. Large differences in results between fields measured 
with static and kinetic perimetry are often seen in patients with larger fields mea- 
sured for moving targets. 

Commercially available field devices such as the Humphrey field analyzer (HFA) 
are not typically appropriate for assessing fields in prosthesis recipients for a number 
of reasons including limitations on target size, intensity, and the relatively limited 
(40 degrees) central region tested. However, a few means of field assessment for 
those with low vision have been developed and are considered in a later section. 

One may question the value of visual field measures in the presence of a visual 
prosthesis. Surely field results can readily be predicted based on the dimensions of 
the electrode array (in degrees), the magnification or minification of the image pro- 
cessing unit, the density of electrodes, and the array location (for retinal implants 
particularly but cortical implants as well). Whether this holds true within the area 
"covered" by the prosthesis remains to be seen. One report indicated that some indi- 
viduals implanted with the Artificial Silicon Retina had larger fields post-op, but that 
others showed shrinkage due to complications [53]. Certainly, field measurements 
are essential for describing (residual) vision outside of this region that will contrib- 
ute not only to field dimensions but also to task performance. Measurement of visual 
fields requires stable fixation at a known (pre-determined) location. For individuals 
with poor vision, placement of a finger at the fixation location is extremely helpful. 

Visual acuity is an index of the finest discernable detail. 6 Visual acuity is typically 
measured using targets approaching 100% contrast (black and white) because 



5 See [14, 48] for a detailed discussion of acuity measurement. 
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resolution improves with contrast. As noted earlier, visual acuity has for some time 
been the predominant outcome measure in intervention studies, and is the primary 
visual descriptor of participants in vision studies, and of populations [24]. At the 
coarse end, visual acuity is clinically described in terms of whether or not the indi- 
vidual can detect hand motion (HM) or count fingers (CF) at a specified test distance. 

It is preferable, of course, to use targets with more precisely controlled and 
defined specifications. Gratings and optotypes are the most common types of acuity 
targets. Grating targets measure the minimum separable resolution whereas opto- 
type acuity is a form of recognition acuity. The relative value of measuring grating 
and optotype acuity is a matter of some debate, and also a matter of circumstances. 
Grating acuity can be quantitatively related to optotype acuity in visually normal 
individuals, but this association breaks down when disease is present (e.g., [32, 36, 
1 10]). When using grating stimuli, aliasing associated with under-sampling or other 
distortions associated with abnormal retina or the prosthesis a concern and can lead 
to an over-estimate of resolution. Aliasing is the situation in which a high spatial 
frequency target is miss-perceived as a stimulus of lower spatial frequency or a 
distorted grating [17, 94, 95, 113, 114]. 

Optotype acuity has won out in clinical settings. Common optotype targets are 
simple shapes, letters, numbers, the tumbling E (formerly "illiterate E"), and 
Landolt rings (also called Landolt C's). The smallest optotype target size (in terms 
of visual angle) that the patient can identify is determined [98]. For the tumbling E 
targets, the observer's task is to indicate in which of the four cardinal directions 
direction the "tines" are pointing. The Landolt C target is a circle with a gap in it. 
The gap is presented in four or eight locations (the four cardinal plus the four 
obliques) and the observer's task is to indicate the location of the gap in each ring. 

Provided that the subject is required to continue to attempt to identify or guess 
until some criterion is reached (e.g., three out of five optotypes are identified incor- 
rectly), acuity measures are criterion and bias free. It is recommended that acuity 
be scored letter-by-letter rather than line by line [15]. 

Though standard, commercially available letter charts such as the ETDRS acuity 
chart [34] or the Bailey-Lovie Chart [16] were not designed to measure extremely 
poor acuity, the lower end of their range can be extended into the range of interest 
by simply decreasing the test distance. At the standard (20 ft. or 6 m) test distance, 
the largest letters on the Bailey-Lovie Chart correspond to an acuity of 20/125; at a 
10 ft. test distance 20/250, and down to 20/2,500 at 1 ft. 

Tests specifically designed to measure acuity for low vision are discussed in a later 
section. An important gain from using optotypes to measure acuity is that the pres- 
ence of measurable optotype acuity provides evidence of form vision capability. 



20.4.1.1 Contrast Sensitivity (Contrast Detection) 

Contrast (of a grating; Peak-to-peak contrast or Michelson contrast) is defined as 
C =(L -L . )I(L +L . ). 

m ^ max min / v max min / 
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Contrast can vary from to 1, and is more often specified as a percentage 
(0-100%). Contrast sensitivity is the inverse of contrast at threshold. Within the linear 
systems approach to vision, a description of an individual's contrast sensitivity func- 
tion (CSF, i.e., the minimum contrast required to see a grating, measured as a function 
of spatial frequency, or bar width) provides a means of knowing the visual system's 
response to any stimulus defined by luminance contrast. For practical purposes, 
though, contrast sensitivity testing is typically limited to a single large (relative to 
acuity) target size, specifying one point on the CSF near the peak of the CSF. 

In clinical settings, optotype measures are more commonly used than grating 
targets [9]. Unlike grating stimuli, optotype targets, such as those on the Pelli- 
Robson Chart [76] are specified in Weber contrast, which is defined as 

C =(L -L )IL . 

w *- max mm ' max 

Note that the two measures are the same if the mean grating luminance is L 12. 
Contrast sensitivity deficits are present in RP patients, even in those with normal or 
near-normal acuity [4, 7]. 

Despite a strong correlation between contrast sensitivity and visual acuity, one 
cannot predict one from another on an individual basis [44], and therefore, both 
should be measured. An individual with very poor acuity, but fairly good contrast 
sensitivity and fields of reasonable size will have no trouble navigating and moving 
through the environment but probably will not be able to read well or at all. The 
converse is also true. An individual with a very small visual field, good contrast 
sensitivity and good acuity will have great difficulty moving about the world or 
finding targets; however, once the targets are "found" (are placed within the func- 
tional field) they will have no trouble identifying the target or reading print. 

Reports suggest that contrast sensitivity better predicts performance than 
other measures (e.g., acuity). Associations have been reported between contrast 
sensitivity and reading performance [57, 111], ambulation mobility [38, 45, 55, 
66, 99], driving [115, 116], face recognition [75, 109], and tasks of daily living 
[79, 80, 109]. 



20.4.1.2 Contrast Discrimination 

Most natural images contain both high and low contrast. In scenes, features to be 
detected are frequently observed in the presence of other supra-threshold (visible) 
background structures. Detection of such features requires contrast discrimination, 
which is necessary for the subsequent process of object recognition. Contrast dis- 
crimination is impaired in RP patients, even those who have good acuity to moder- 
ately reduced contrast sensitivity [6]. 

No simple chart or other test of contrast discrimination is available though, in 
principle, one could be developed fairly easily. Such a chart might consist of sets 
of stimuli each with at least two elements ranging in contrast, with the patient's task 
being, for example to identify the stimulus with the highest contrast, with the 
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difference between the contrasts among stimuli in stimulus sets decreasing down 
the chart. PC-based tests on the same principle would be more flexible. 



20.4.1.3 Motion Perception 

The ability to detect a luminance-defined, or color-defined target as moving and to 
judge motion speed and direction requires detection and localization over an 
extended retinal area, as well as interactions between neighboring areas and, intact 
temporal processing. Human visual motion occurs beyond the retina [12], so that 
intact temporal processing at earlier stages is crucial. The temporal sequence of 
retinal stimulation in a degenerated retina may become highly distorted, so that the 
signals reaching the cortex become ambiguous [25]. Temporal processing and 
displacement discrimination are abnormal in RP patients [3, 5]. Prosthesis pro- 
cessing delays and those from degenerating retina may favor slow stimuli of 
coarse spatial grain. Movement perception is likely to be greatly impaired in retinal 
degenerations [25]. 

In fact, perceived motion in the presence of a prosthesis is most like apparent 
motion, or sampled motion, which is the perception of smooth motion from sequen- 
tial presentation of discrete stationary targets, in this case, electrode-generated 
phosphenes. The appropriate combinations of temporal and spatial interval charac- 
teristics for apparent motion have been worked out for normally sighted individuals 
[19]. However, the same relationships are unlikely to hold for prosthetic vision. 

Bearing these considerations in mind, motion processing may thus be severely 
impaired in (potential) prosthesis recipients. The loss and rewiring of post-receptoral 
elements is an additional factor in the case of sub-retinal implants. 

In patients with retinal degenerations who retain relatively good form vision, 
aspects of motion perception that have been assessed include judgments of motion 
displacement thresholds [5] and heading direction [101]. Minimum displacement 
thresholds are increased and maximum displacement thresholds decreased, greatly 
restricting the range of detectable motion in patients with RP, even when visual 
acuity showed only minor reductions (acuity of 20/40 or better) [5]. Patients with 
retinal degenerations with form vision have elevated thresholds, reduced maximum 
velocity and/or direction discrimination for two-dimensional (2D) motion. Yanai 
et al. [118], testing three RP patients implanted with a 16-electrode prosthesis, 
found that performance on a motion discrimination task was above chance only so 
long as the subjects were allowed to move their heads to scan. 

2D motion perception imparts important information unrelated to perception of 
motion per se. For example, motion parallax is an important depth cue, particularly 
for those lacking binocular vision, among whom are prosthesis recipients. Object 
(or person) motion facilitates detection, particularly in complex environments. 

In terms of survival skills, the ability to judge motion toward or away from one- 
self may be more important. Our movement through an environment is guided by 
optic flow, perceived visual direction, and judgment of focus of expansion, among 
other things. Turano et al. (2005) reported a change in the ability of individuals with 
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field changes to utilize optic flow for judging heading direction, important for 
orientation and mobility [101]. It is equally important to judge a moving object's 
path with respect to oneself (to avoid collision or catch a ball). This is based on, for 
example, looming and zooming cues, the detection and interpretation of which may 
be greatly impaired in prosthesis recipients. 

Assessment of the many aspects of motion perception is unpractical. Further, 
many aspects of motion perception require some level of spatial vision that may not 
be met by the prosthesis patient. A practical test of motion direction discrimination 
might be to use a bright large dot or line moving on a dark screen in a dark room 
along one of four (cardinal directions) or eight (with diagonals) and requiring that 
the subject identify the direction of motion. A similar approach using a penlight 
instead of a dot can be used in cases of high light thresholds an has been shown to 
have good reproducibility within intact visual field areas [49]. However, it is uncer- 
tain that such a test will tell us much about many aspects of motion. Tests of seem- 
ingly unrelated aspects, such as the presence of long range interactions and the 
ability to judge the relative timing of flashes may be much more informative, as 
both are prerequisites for perception of motion. 

Long-range spatial interactions are key to integrating information within a 
scene and detecting motion. The most striking demonstrations of long-range spatial 
interactions are illusory or subjective contours [52], examples of which are shown 
in Fig. 20.2. As can be seen on the right, a central inverted triangle appears though 
there are no lines to demark it. Its presence is induced by the corner elements, and 
despite its lack of true form, it is seen as occluding the upright "triangle" inferred 
from its corners. The simpler form of the induced contour is shown in the left half 
of Fig. 20.2. Perception of these contours demonstrates the presence of the capacity 
for long-range interactions necessary for motion perception as well as judgments 
such as figure ground. These functions serve many important purposes in making 
sense of visual scenes. Long-range interactions are also necessary for recognizing 
partially occluded objects in a cluttered environment. 



* T ▼ 

Fig. 20.2 Examples of illusory contours produced in the presence of elements that appear par- 
tially occluded. Left. One illusory square. Right. Two illusory triangles 
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20.4.1.4 Depth Perception 

Stereopsis is the perception of depth produced by binocular retinal disparity. 
Stereopsis subserves fine depth discriminations for near objects. Prosthesis recipi- 
ents will not have stereopsis. These individuals will necessarily rely on monocular 
depth cues including those mentioned previously (occlusion and motion parallax), 
as well as lighting and shadows, linear perspective, texture gradient, height in field, 
to name but a few. 

Color is one of the most compelling aspects of vision, adding beauty to our 
visual experience as well as aiding in detection and identification of objects. Some 
reports indicate that induced phosphenes are of many colors [51, but cf. 31]. 
Nonetheless, due to the complexity of color-conferring circuitry and crudeness of 
available stimulation methods, veridical color vision will not be afforded by pros- 
theses in the foreseeable future. 



20.4.2 Tests Used in Prosthesis Trials 

In the absence of a well-described, validated set of tests, measuring visual function 
has been and still is up to the ingenuity of those working on the projects. Dobelle 
(2000) measured visual acuity in three recipients of their 64-element cortical 
implant and reported acuity and visual field size for one [31]. For this subject acuity 
measured using tumbling E's and Landolt C's show excellent agreement (20/1,200). 
Second Sight Medical Products (Sylmar, CA) has published a report on visual func- 
tion in three recipients implanted with their 16-electrode prosthesis [117] based on 
three to four alternative tests (locate and count objects such as common household 
items, discriminate among those items, determine the orientation of a large L, and 
identify the direction of object motion). A grating orientation discrimination task 
was used to measure acuity. Kiser et al. used standard static (HFA to test central 30 
degrees) and kinetic perimetry (Goldmann to test far periphery) methods to test 
vision of eight Artificial Silicon Retina implant (Optobionics Corp.) [53]. Yanai 
[118] found that, with repeated testing, three implanted subjects could discriminate 
between plate, knife or cup against a dark background. 



20.4.3 Tests that Have Been Designed for Use with Prostheses 

A few laboratories have developed new tests specifically for assessing prosthesis 
recipients, but have not yet tested recipients with them. In many instances the 
devices have been used in the context of simulated prosthetic (pixelized) vision 
(e.g., [26, 27, 87, 96]), but have not been used to assess prosthesis candidates or 
recipients. In addition, simulations in subjects with better vision have been used to 
estimate how well a person may perform using a prosthetic, with the idea that the 
pixels correspond to electrodes [25, 106]. 



404 M.E. Schneck and G. Dagnelie 

20.4.4 Vision Tests for Very Low Vision 

PC -based tests to assess basic aspects of visual function in prosthesis recipients 
have recently been developed. 

The BaLM test (Basic Light, Localization and Motion; Zrenner and Wrobel, 
Retina Implant AG) [112], measures several visual functions, including light detec- 
tion, light location, temporal resolution and motion direction discrimination. All 
tests involve forced-choice responses, provide optional auditory feedback and allow 
the number of trials to be varied. Output includes percent correct and response 
times. 

The Berkeley Rudimentary Vision Test was created by Ian Bailey and co-workers 
to assess individuals with very low vision, within the range typically described as 
hand motion or count fingers (i.e., worse than 20/800 (LogMar 1.6)). It contains a 
light and basic form perception test (BFPT) and an acuity screening and measure- 
ment test, the Single Tumbling E's Test (STET). Visual fields can also be 
measured. 

FrACT, the Freiburg Acuity and Contrast Test [11] is available online at 
http://www.michaelbach.de/fract/index.html. It uses Landolt C's to assess very 
coarse acuity in the range of hand motion to count fingers, and contrast sensitivity. It 
also assesses vernier acuity. By combining the results of FrACT, clinical acuity mea- 
sures, and ETDRS acuity, Schulze-Bonsel et al. found that CF and HM acuity can 
be reproducibly assessed and correspond to acuities of 20/1,400 and 20/4,000 [84]. 

Dagnelie and co-workers have developed a number of PC-based tests appropri- 
ate in this context. These include a visual field measure [13, 28], yet to be validated, 
that can bridge the gap between crude localization and standard field measures. 



20.5 Visual Performance Assessment 

As has been noted [25], it is not what the prosthesis recipient can see, but 
what they can do that is critical. The aim of prostheses is to improve the ability 
of the recipient to perform activities of daily living (ADL), instrumental tasks of 
daily living (IADL) and what we may call activities of life satisfaction (ALS). 
Visual performance affects independence, quality of life and "visual disability" 
[67]. Assessing this aspect of vision remains a major hurdle in the path of 
prosthesis evaluation and progress. Assessment of visual performance (pre- and 
post-implantation) addresses questions such as: How has everyday task perfor- 
mance changed since implantation? Is the increment in performance sufficient to 
be of real value to the recipient? Does the prosthesis increase efficiency, reduce 
risk and increase independence? 

This aspect of visual assessment has two branches: measured performance and 
self-reported performance. Direct measurement has been argued to be superior to 
self- report, at least in aging [43]. Theoretical advantages of measuring function 
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include better reliability and validity, greater sensitivity to change and less influence 
by confounding factors such as culture and language [45]. However, questionnaires 
enable us to assess important issues beyond the realm of performance, such as 
whether the implant was of any benefit (e.g., all of the tasks can now do) to the 
recipient, the difficulty an individual has performing particular IADLs, and 
the impact that difficulty or inability has on quality of life. Questionnaires provide 
valuable information that may guide the choice of performance measurement. Both 
approaches are important for demonstration to patients, care-givers, government, 
and funding organizations the effectiveness of the prosthesis. 



20.5.1 Measured Performance 

Observation of blind individuals who have undergone rehabilitation reminds us that 
there are fewer tasks than we realize that cannot be accomplished without vision. 
However, the return of the ability to perform tasks visually is desirable to those with 
little or no vision - particularly those who, like the subjects in the current implant 
studies, have lost their vision later in life. 

Among tasks that cannot be performed at all without vision, one of the most 
important is reading printed text with all its images, information provided by for- 
matting (e.g., headings, emphasis), potential for scanning to find information, etc. 
(Braille comes in only one "font" and letter size). Other desirable tasks that require 
vision are driving and the ability to identify and locate at a distance (Josh Miele, 
personal communication). 

There is no generally accepted test battery of even rudimentary task performance 
measures for use in low vision patients (including prosthesis recipients). Many of the 
performance batteries that have been developed are not relevant to or have not been 
assessed in those with very low vision (e.g., [46, 54, 93, 102]). In choosing items of 
ADL to assess for their index, Haymes et al. (2001) considered how common the 
ADLs were on existing instruments (questionnaires) and whether they were consistent 
with daily living problems reported by a very large number of people with vision 
impairment [46]. Unfortunately, the tasks chosen which include reading print in various 
contexts (e.g., newspaper, medication label), using a telephone, recognizing faces, 
threading a needle, require vision superior to that currently afforded by prostheses. 

Optimally, performance would be measured in situ. However, it is more practical 
to measure tasks under controlled (laboratory) conditions. In either case careful 
consideration of task relevance is crucial. 

The most commonly used tasks are reading, face recognition, and mobility and 
orientation (wayfinding). As an alternative or in conjunction with measurements of 
ADLs and IADLs, which are complex, one may assess "component" tasks such as 
eye-hand coordination, visual search, figure ground discrimination, and finding 
embedded objects [2, 54, 58]. 

Wilke et al. (2007) developed two sets of test which we shall refer to as "at table" 
tasks and orientation and mobility tasks for use in visual prosthesis recipients [1 12]. 



406 M.E. Schneck and G. Dagnelie 

The at table tasks incorporate an important component of many activities, eye hand 
coordination. The orientation and mobility test uses projected images of a street 
scene as viewed from different distances and notes the "viewing distance" at which 
particular scene items are first seen. Importantly, both tasks incorporate a measure 
of the time required to complete tasks. 

Turano and coworkers have developed both real [99] and virtual [100] environ- 
ments in which to assess mobility performance. Velikay-Parel et al. (2007) have 
developed a mobility task for individuals with very low vision [104]. 

One of the authors (GD) has developed a set of task performance measures 
beginning with a search task, locating and counting white checkerboard squares 
[27, 106]. This activity is followed by a measure of eye-hand coordination: placing 
black checkers on the white checkerboard squares they had previously counted. 
Scoring is in terms of time to complete the task (speed) and the number of checks 
that are not or are incompletely covered (accuracy). Another test measuring eye 
hand coordination is a maze tracing task, scored in terms of speed and accuracy (or 
rather errors: cumulative area spanned by tracing outside the borders) [71]. In addi- 
tion, a complete record of the performance is recorded for more detailed analysis. 
To date, these tests have only been used to assess simulated prosthetic vision 
(coarsely pixelized vision). 

Wayfinding by individuals with poor vision is of great interest to low vision 
researchers. Wayfinding includes mobility ability, orientation skills, the ability to 
form mental maps or learn a route. Because of safety concerns focus has been on 
two scenarios: visually guided travel in the laboratory and cane-assisted travel in 
everyday environments. A difficulty facing this area of research is that individuals 
learn test routes quickly, limiting the number of "trials" that can be used. Real- 
world wayfinding is subject to numerous uncontrollable variables and may require 
the presence of an O & M instructor, further limiting its practicality. There is a 
continuing effort to overcome these problems (e.g., [56, 104]). Velikay-Parel 
et al. (2007) addresses the difficult issue of repeatability of the measures [104]. 

Evaluation of task performance is based on speed and accuracy. These are easily 
quantified for simple tasks. However, for more complex tasks, such as ADLs and 
IADLs, an occupational therapist and orientation and mobility trainer, masked as to 
whether the individual has received a prosthesis, may better evaluate task performance. 

In summary, measured task performance brings us closest to knowing the benefit 
gained by the individual with respect to everyday activities. Establishment of a battery 
that is both relevant and appropriate for those with very low vision, and validating and 
standardizing such a battery would be of enormous value to the field of prostheses. 



20.5.2 Self-Reported Performance (Questionnaires) 

Funding organizations such as the NEI require the inclusion of a patient-reported 
outcome for clinical trials of any disease intervention or treatment or assistive 
device, and the FDA also considers information from visual function questionnaires 
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(VFQs) [24, 77]. Instruments have long been the means of assessing the success or 
failure of rehabilitation programs (e.g., [10, 29, 88, 90]). 

The rising use and interest in questionnaires has been driven by both need and, 
largely, by improvements in the methods of development and data analysis. The 
latter charge was lead in large part by Robert Massof . He applies techniques including 
Item Response Theory and Rasch analysis that have greatly increased the value and 
"interpretability" of the information gathered. Massof (2007) has quite clearly laid 
out the merits of and need for these techniques [68]. 

Visual function questionnaires contains activities to which difficulty ratings 
(related to vision) are assigned by study subjects/patients. There is an abundance of 
VFQs, including the NEI VFQ-25 [62], VAQ (visual activities questionnaire) [86], 
and Veterans Affairs questionnaires [89] for low vision (VA LV VFQ-48 and LV 
VFQ-20). 

There are also many instruments addressing Activities of Daily Living (e.g., 
[46, 63]). 

More recently, instruments assessing Health-Related Quality of Life (HR-QOL) 
or Quality of Life (QOL) have come into use. The content of HR-QOL question- 
naires typically includes assessment of the ability to perform tasks of daily living, 
interactions with other people, emotional well-being and independence [33]. 

The broad range of instruments available, with varying content, and some devel- 
oped before and some after the changes in design and scoring alluded to above, 
makes choosing the appropriate instrument tricky. The means to assess the quality 
of an instrument and to decide whether it is appropriate for use in a particular con- 
text as well as key issues for questionnaire development have recently been nicely 
laid out [77]. 



20.6 Summary 

Assessing the efficacy of visual prostheses is a complex undertaking. There are 
many issues to consider in the assessment of visual prostheses. As the technologies 
continue to evolve, there will be a changing dynamic involving the steadily improving 
capabilities of the technology and the unique needs of the growing number of target 
populations. What is common in all circumstances is the need to assess visual func- 
tion, visual performance, and self-perceived visual ability using tasks that are 
appropriate to the target population in terms of functional level and needs, with 
methodologies that meet generally adopted criteria scientific rigor, including ade- 
quate pre- and post-implant evaluation and appropriate control procedures. 
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Chapter 21 

Activities of Daily Living and Rehabilitation 

with Prosthetic Vision 

Duane R. Geruschat and James Deremeik 



Abstract Now that technology has the capability to provide ultra-low vision to 
individuals who are functionally blind, there is a recognized need for vision reha- 
bilitation to become part of the process of adaptation. This chapter will present 
concepts of rehabilitation as they relate to prosthetic vision, describe approaches 
to evaluation and instruction, address issues related to measuring outcomes, and 
offer thoughts on the future of rehabilitation for individuals with prosthetic vision. 
The purpose of this chapter is to describe the challenges and opportunities of 
prosthetic vision in the context of using such vision for activities of daily living and 
to propose rehabilitation techniques that could assist patients as they adapt and inte- 
grate prosthetic vision into their lives. The chapter will be divided into four sections: 
Concepts of Functional Vision and Rehabilitation, Evaluation and Intervention with 
Prosthetic Vision, Measuring Functional Outcomes, and The Future. 
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21.1 Concepts of Functional Vision and Rehabilitation 
21.1.1 Application to Orientation and Mobility 

Ultra-low vision is at the lower end of the clinical visual acuity continuum, which 
includes light perception, light projection, and form perception; it can have a func- 
tional impact on individuals with visual impairments by enhancing or improving 
their orientation and mobility (O&M) skills. For example, when walking the halls 
of a residential school for students who are blind it is not uncommon to see two to 
three totally blind students holding the arm of and following behind the one student 
who has form perception. The lead student, using form perception, can see the 
lights in the ceiling and visually trails the lights to maintain a straight line of travel 
down the corridor. Another utility of ultra-low vision is demonstrated by the fully 
sighted person who wakes in a hotel room in the night and uses the moonlight shining 
through a gap in the curtain or the ambient light from the alarm clock to orient 
himself and locate the entry to the bathroom without turning on a light that might 
disturb his spouse. 

The ability of a person with ultra-low vision to visually detect contrast can 
enhance her awareness of her location in a room. The left panel of Fig. 21.1 shows 
a white door in a white room (low contrast); the right panel shows the same door 
but with a dark-colored robe on the door's hook (high contrast), which makes the 
door easier to identify visually. In this example, a simple environmental feature (the 
placement of a robe) can enhance movement through the room for a person with 
ultra-low vision. The benefits afforded by the ability to perceive light or see con- 
trasting colors illustrates why we believe that prosthetic vision can be useful for 
orientation and mobility (O&M). 




Fig. 21.1 Effect of contrast on visibility: a dark robe on a light door 
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Although there are a variety of technological approaches to providing an indi- 
vidual with prosthetic vision, when the technology allows the individual to reverse 
contrast, this feature may enhance the individual's ability to detect objects and the 
like. Many patients who utilize a closed circuit television (CCTV), for example, 
prefer to do so by making the letters white and the background black (that is, by 
reversing the contrast of the monitor). This technique could be applied to mobility, 
for finding a doorway out of a well-lit room, if the individual using prosthetic 
vision reversed contrast to show a bright door opening in a dark wall. 

The foregoing examples described potential enhancements to orientation with 
no descriptions of benefits to mobility, because the current prosthetic vision tech- 
nology is not sufficient to replace or eliminate the need for a long cane or a guide 
dog for independent travel in unfamiliar environments. The point is that ultra-low 
vision can have a positive impact on functional orientation but not on mobility in 
novel environments. 

Today's prosthetic vision may be potentially safe enough for an individual to 
use it as their primary source of mobility information (no cane or guide dog) 
indoors in a controlled and familiar space or when locating furniture or objects 
with high contrast. A "controlled space" is an indoor environment in which 
changes in elevation (stairs) are not present or their location is known, and in 
which furniture and other room elements maintain the same location over time. 
Travel in unknown and/or complex environments (crossing the street or walking 
in a shopping mall) requires the use of a long cane or guide dog. In such a situa- 
tion, prosthetic vision can be used as a supplementary source of information to 
enhance the individual's orientation while other sensory information (audition, 
tactual) is combined with primary sources of information for mobility: the long 
cane or guide dog. 



21.1.2 Application for Activities of Daily Living 

Because prosthetic vision provides very low levels of visual acuity, activities of 
daily living (ADLs) that require detailed vision (sewing, reading, or the recognition 
of facial features) are not envisioned as being amenable to prosthetic visual reha- 
bilitation until the level of resolution the technology provides has been substantially 
improved. The opportunity presented by the current technology, which allows users 
to perceive high contrast can be of benefit with a variety of ADLs, including per- 
sonal care and personal management. For example, in the area of personal care, the 
ability to identify toothpaste on a toothbrush might be accomplished with the use 
of high contrast (green toothpaste on a white-bristled toothbrush). Visually locating 
soap or a shampoo bottle in a bathtub may be possible with high contrast. The ability 
to apply lipstick may be enhanced with ultra-low vision. The ability to visually sort 
dark- and light-colored socks, to identify a white shirt from a dark-colored shirt 
may be possible with prosthetic vision. The use of contrasting colors in the kitchen 
could prove to be beneficial for people with prosthetic vision, who may be able to 
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Fig. 21.2 Effect of contrast on visibility: a dark placemat and a light dish 



differentiate milk from juice or mayonnaise from ketchup in the refrigerator. The 
use of high contrast between a dark placemat and a light-colored dish can enhance 
a client's ability to locate the dish, as illustrated in Fig. 21.2. 



21.1.3 Patient Lifestyle and Expectations 

Early chapters of this book concentrated on the visual system to the exclusion of 
personal history. As rehabilitation specialists, we think of vision in the context 
of the person, their history, lifestyle, expectations, and acknowledge that these 
personal elements influence the way vision is used. Consider two patients who 
have the same clinical vision status (visual acuity, contrast sensitivity, visual 
field); one patient uses a cane and minimizes the use of their remaining vision, 
and the other travels without a cane, and utilizes optical equipment to read street 
signs and view traffic lights. Individuals who maximize the use of their remain- 
ing vision (light perception) prior to implant tend to have the best prognosis for 
integrating prosthetic vision into their lifestyle and to experience more benefits 
after implantation. 

The management of patient expectations is a key element to successful func- 
tional outcomes with prosthetic vision and must be considered part of the rehabili- 
tation process. Patients want to know how their life will change or improve with 
prosthetic vision. Will prosthetic vision resolve their functional problems? 
Research on the most common functional problems in mobility clearly shows that 
managing illumination (light adaptation, low-light environments), detecting 
changes in elevation such as drop-offs (curbs, stairs), and crossing the street are 
three of the most common problems for patients with low vision [4]. Our expe- 
rience with prosthetic vision suggests that these leading low vision mobility 
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problems may not be addressed by the current technologies. The implant wearers 
we have seen at the time this chapter is being written are, in the context of O&M, 
quite similar. They are all independent travelers who use some combination of 
long cane, guide dog, and/or remaining vision. They travel in familiar and unfa- 
miliar areas, ride public transportation, and do not report a serious limitation to 
their independent travel because of the loss of vision. Because the prosthetic 
vision systems we have worked with provide ultra-low vision, we have not identi- 
fied anyone for whom prosthetic vision has been sufficient to replace the long cane 
or the guide dog when walking in unfamiliar areas. Until the technology improves 
visual acuity and visual field, prosthetic vision is viewed as an additional travel 
aid, an enhancement to travel, specifically orientation, rather than a substitution 
system that would supplant the primary travel aid. Therefore, specialized instruc- 
tion from properly trained rehabilitation professionals will benefit a prosthetic 
vision program. 



21.1.4 Congenital and Adventitious Vision Loss 

We assume there is an interest in offering prosthetic vision to those with congenital 
blindness. There is a significant difference between the visual abilities and the 
psychological adjustment process of someone with adventitious vision loss who has 
had his sight restored (cataract extraction, corneal transplant) and an adult with 
congenital vision loss who has been provided sight for the first time as an adult. 
Personal accounts such as the experience of Mike May, described in the book 
Crashing Through [7], show that clearing the optical pathway does not result in an 
immediate improvement in functional vision for someone who is congenitally 
blind. In fact, the more common experience involves a sense of being overwhelmed 
and confused [6]. It is important to recognize that an individual who has lived as a 
blind person does not suddenly benefit from visual input. If the patient has a con- 
genital vision loss, a significant period of adaptation, learning to interpret this novel 
sensory input and time to develop a visual memory, will be required. If the patient 
lost vision later in life, the age when it was lost, their ability to use low levels of 
vision as their vision gradually decreased, the primary learning modality for gathering 
information from the environment (visual, tactual, auditory), and the amount of 
remaining visual memory are a few of the issues to consider before implantation. 
These issues may also become a part of the screening and selection process for 
those who will participate in any type of prosthetic vision rehabilitation program 
because different strategies may be needed when providing rehabilitation training 
to these individuals. 

Another often repeated description from those who gained sight as adults 
involves the amount of effort required to process visual input. Mike May describes 
the need to close his eyes to process information and to feel calm [7]. As revealed 
in May's experiences, there are unknown challenges that await the patient who 
gains sight after leading the life of someone who is congenitally blind. 
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21.2 Evaluation and Intervention with Prosthetic Vision 
21.2.1 Evaluation 

The process of rehabilitation always begins with an evaluation. The evaluation 
of prosthetic vision should begin with an assessment of functional vision [2, 8, 
9]. The typical functional vision assessment is hierarchical and begins with 
evaluating the ability to respond to a light source, determining if the patient can 
localize, fixate, track, and scan the light. This is followed by an assessment of 
visual motor skills then higher order perceptual skills (color identification; 
three- and two-dimensional shape recognition; and symbol, letter, and word 
recognition). 

The standardized functional vision assessment is followed by a comprehensive 
evaluation of the use or non-use of vision for mobility [4], personal care, and/or 
personal management. A few examples of this assessment include the ability to 
detect changes in contrast of open doorways, the ability to locate windows, to visu- 
ally trace the lights in a hallway, and the ability to identify the location of a white 
napkin on a dark table. 

Information gained from such assessments can be useful for developing an inter- 
vention program. For example, let's assume one patient with no residual vision 
travels independently without a long cane or guide dog, but only in her home and 
within a fenced back yard. The patient has small children and frequently steps on 
or kicks toys and bicycles that are left on the floor or grass. During the evaluation 
it is determined that the goal for this patient is to improve her ease of travel in the 
home and around the yard through improving her ability to locate high-contrast 
objects. Prosthetic vision (form perception) could enhance this patient's life by 
reducing the frequency with which she kicks her children's toys. The intervention 
may involve teaching the patient scanning techniques to locate obstacles. It may 
also be necessary to teach the patient how to interpret prosthetic vision, to essen- 
tially determine the identity of the low-resolution images her prosthetic vision 
provides. This patient's vision rehabilitation program will emphasize integrating 
prosthetic with other sensory information to reduce the mental effort that occurs 
when she experiences a new type of sensory input. 

Another patient with minimal residual vision travels independently on public 
transportation using a long cane supplemented by light projection. Prosthetic 
vision for this patient may provide form perception and the ability to differenti- 
ate areas of high contrast such as the sidewalk (light) from the pavement of the 
street (dark). This patient would be introduced to the same concepts that are cur- 
rently taught to someone with severe low vision. Specifically, instruction would 
address the issue of when it is safe to use vision only and when vision is only 
safe to supplement the use of the long cane. This is one of the more challenging 
skills to acquire. It is difficult to always know when vision alone can be the 
primary modality and provides sufficient information for making decisions 
about safety. 
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21.2.2 Intervention 

As the technology improves, allowing for sharper visual acuity, contrast sensitivity, 
and an increase of peripheral visual fields, we expect that some type of instruction 
to enhance the use of prosthetic vision will be beneficial. We assume there would 
be two distinct approaches to instruction, one approach for patients who present 
with congenital blindness (no visual memory) and an approach for patients who lost 
vision later in life. 

The descriptions from prior decades on the effect of adult onset of vision could 
be instructive for anticipating and understanding the challenges to be faced by an 
adult with late onset of vision [3], We would expect to observe a patient with lim- 
ited ability to comprehend what they were seeing. The work of Mary Anne Frostig 
and Natalie Barraga during the 1960s [1], specifically their instructional procedures 
that follow the process of visual perceptual development, would be a useful place 
to begin instruction. 

The large body of literature on instructional strategies for children with low vision, 
as well as the literature on visual perceptual instruction that has evolved during the 
past 40 years, provides useful concepts and instructional sequences that could be 
adapted for an adult population. Since the adult patient with congenital blindness and 
adult onset of vision is functioning visually at an earlier developmental level, materials 
written for children may prove to be useful. The American Printing House for the 
Blind has a collection of materials such as Bright Sights that are designed primarily 
for children with low vision. The materials provide lesson plans of sequential lessons 
as well as assessment tools to monitor progress. For use with adults, the materials and 
lessons would need to be modified to be age appropriate. Isolating the visual system 
in the early developmental stages before integrating visual information through a 
multisensory approach has been demonstrated to be effective with young children. 
Experience will be required to determine if this same approach would be useful for 
adults with congenital blindness and adult onset of vision. 

For adults who have visual memories, early visual developmental skills should 
still be present or could be re-acquired fairly quickly. Intervention strategies for 
these adults could include the introduction of specific visual skills such as estab- 
lishing a consistent response to a given visual stimulus (type of light source), sys- 
tematic scanning to localize objects, fixate on the object, tracking and shifting gaze 
between objects, as well as some perceptual training to learn to (re)interpret the 
visual world. Instruction may also involve teaching low vision skills such as scan- 
ning to define the borders of an area, practicing the important ability of being sys- 
tematic, and scanning for objects in the direction perpendicular to their primary 
orientation [4]. 

Localizing and fixating are specific techniques that can be impaired by ocular 
pathology. We do not know the effects of prosthetic vision on these skills. There 
may be a need to introduce eccentric viewing to improve visual clarity, as well 
as the concept of turning the head to an eccentric position to improve visual 
ability. 
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Tracking and shifting gaze are important skills for personal care and mobility. 
For example, applying make-up involves eye-hand coordination and the ability to 
shift gaze from a cosmetic to the reflection of one's face in the mirror. In mobility 
the ability to track a moving vehicle to assess time-to-contact (impact) and shift- 
ing gaze to acquire information at a two-way street crossing are common mobility 
tasks. 



21.3 Measuring Functional Outcomes 

It is important to determine the effects of prosthetic vision on functional perfor- 
mance. Did the treatment make a difference? If so, what kind and how much of a 
difference? Is there a difference in the patient's posture, body position, and/or head 
position? Did the treatment result in greater safety or was the level of independence 
improved? Does the patient experience greater visual independence when preparing 
dinner? Have additional responsibilities in the home been absorbed by the patient 
such as sorting laundry or selecting the proper placemat and plate while setting the 
dinner table? 

It is critical to understand what each patient's goal is for prosthetic vision. If the 
goal is for prosthetic vision to replace or supplement the individual's need to travel 
with a long cane, then the resulting prosthetic vision will need to be of sufficient 
resolution to allow the person to reliably and safely detect changes in elevation 
(curbs, stairs), confidently travel in a variety of light levels, accurately detect 
objects in the travel path, and consistently identify changes in the travel surface 
(gravel, grass, pavement). If the goal is simply to enhance the currently existing 
travel skills, then the resolution provided by the prosthetic vision can be more 
modest. 

Another prosthetic vision recipient may have sufficient functional vision to 
travel safely in familiar indoor and outdoor areas, but use a long cane when traveling 
in unfamiliar areas. In this example, the question is what type and how much pros- 
thetic vision will be required to improve travel skills (light perception to light 
projection or hand motion)? 

Assuming the subject has some amount of independent mobility prior to the 
treatment, the effect will either be to change the mode of travel (non-visual to 
visual) or to enhance the current approach to travel. Herein is the challenge. If the 
patient is an independent traveler prior to the introduction of prosthetic vision, 
prosthetic vision may not increase their level of independence. The best possible 
outcome would be an enhancement of independent travel, that is, improved orien- 
tation in unfamiliar areas. It is quite difficult in general to measure the enhance- 
ment of travel. If the expected outcome of prosthetic vision is to change the mode 
of travel (cane or guide dog to vision only), the technology is not yet capable of 
providing that level of visual input. Both examples present challenges to measur- 
ing outcomes. For example, if the patient is an independent cane traveler or guide 
dog user, taking the cane or dog away may result in a degradation of performance 
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until the new visual approach has been mastered, assuming the future of prosthetic 
vision provides for better visual acuity and visual field. In this example, a degrada- 
tion of performance would be expected for the short term with the hope that per- 
formance will improve as mastery of prosthetic vision occurs. However, even in 
this example, the best we can hope for is for the patient to transition from being 
an independent blind traveler to an independent visual traveler. When vision is 
considered as an enhancement to the current mode of travel the measurement chal- 
lenge is more extreme. Attributes such as ease, anticipation, or previewing are 
difficult to quantify. 

One approach to measuring mobility performance would be to isolate the visual 
aspects of mobility and to concentrate testing on these visual tasks. For example, 
walking down a hallway while following the ceiling lights could be compared 
to walking down the hallway without the ceiling lights, identifying the location of 
windows in a room, or visually identifying the fourth intersecting sidewalk are all 
discrete elements of O&M that can only be done visually. 

Another approach to measuring mobility performance with prosthetic vision is 
to measure mental effort. The underlying assumptions of this approach are: 

1 . Patients with low vision rarely bump into obstacles 

2. It requires more cognitive attention to the environment to travel with no vision 
than low vision 

3. This attention can be measured as mental effort 

Experiments have shown that measures of mental effort through the use of a 
secondary task are responsive to variation in environmental complexity [10] and to 
varying extent of visual field [5]. Assessing mental effort may be an approach with 
potential for measuring outcomes with prosthetic vision. We assume, however, that 
the introduction of prosthetic vision would itself impose a secondary task and at 
least initially result in an increase of mental effort until the patient adapted to the 
new visual input. We have observed patients with recent implants whose perfor- 
mance is initially degraded as they adjust to the prosthetic vision. At times we have 
also observed that patients ignore other sensory information as they strive to utilize 
their new vision. Time is required for patients to complete their adjustment to the 
prosthesis and reintegrate all the sensory information it provides. 

In the context of ADLs, the issue of independence of people who are blind is 
also present. Since many patients can perform ADLs without sight, it is challenging 
to measure the effects of a prosthetic implant on living skills. One approach is to 
isolate the visual elements of a task and to concentrate the performance measure on 
those specific elements. For example, the identification of a white shirt may be 
done factually via the feel of the cloth, the location of the shirt in a closest, or a 
tactual marking on the inside of the collar. However, it may also be possible to sort 
shirts based upon visual input, separating white shirts from dark shirts, and this is 
what should be evaluated in people with prosthetic vision. Another example is the 
height of flame on a stove. The common approach is to attach tactual markers to 
the stove dials that indicate the relative height of the flame. Prosthetic vision may 
be used to visually detect the flame. 
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21.4 The Future 
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Although prosthetic vision does not currently offer those who receive it much assis- 
tance with tasks that require good functional visual acuity, we anticipate that the 
next generation of this technology, if it offers a modest improvement in visual acu- 
ity, may be of benefit for tasks that require high contrast and do not require good 
visual acuity. For example, a black hairbrush on a white countertop in the bathroom 
might be located through the use of artificial sight. In the kitchen, counting and 
placing strawberries on a white cutting board, working with peeled white potatoes 
on a dark cutting board, locating food such as beef on a white plate, determining 
how much liquid is in a glass (see Fig. 21.3), and sorting laundry into dark and light 
piles are activities that could be aided by the use of artificial vision. 

Anticipating that the technology will improve over the next decade, we would 
expect that tasks requiring higher levels of visual acuity will become possible 
through the use of prosthetic vision. 

Prosthetic vision has been described as being analogous to cochlear implants. 
Although this analogy may prove to be accurate, we provide a different analogy that 
offers a word of caution. Electronic travel aids (ETAs) have been designed to pro- 
vide information about the environment with the goal of offering an improved 
preview of the environment. Common examples of ETAs include the laser cane and 
the sonic guide. These two devices provide detailed information about the environ- 
ment. Users of such devices find that they compete with the naturally occurring 
sensory information and tend to only use them in specific and isolated situations. 
For example, the sonic guide is effective for following the barrier that separates 
paid customers from the general public in a subway station, allowing the user of the 
sonic guide to follow the barrier without touching it. In other situations such as a 
grocery store, users describe the overwhelming amount of information it provides 
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Fig. 21.3 Effect of contrast on visibility: dark liquid in a clear glass 
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as competing with the naturally occurring sensory input. Mike May's descriptions 
of his experience with regaining vision are similar to the reactions of users of the 
sonic guide in sensory-rich environments. We wonder if improvements in the level 
of resolution of prosthetic vision will result in clients receiving too much visual 
information to process, inhibiting their functional performance. 

As the quality and quantity of prosthetic vision improves, we assume that 
patients who are congenitally blind will receive implants in greater numbers. The 
history of adult-onset vision, as previously mentioned, suggests there will be chal- 
lenges. A few of these may include 

• Learning to interpret the image 

• Adapting to the new sensory input 

• Integrating vision into a lifestyle of non-visual independence 

We believe it will be necessary to educate low vision rehabilitation service 
providers in how to work with individuals who have prosthetic vision. We do not 
know if the standard low vision rehabilitation techniques will apply to this popula- 
tion or if entirely new strategies will need to be developed or if the modifications 
to the existing approaches is all that will be required. Low vision rehabilitation 
professionals working with recipients of prosthetic vision will need to be familiar 
and competent with non-visual techniques and strategies in the performance of 
ADL tasks as these skills may continue to be essential for the individual with pros- 
thetic vision. We have emphasized addressing mobility skills with this population, 
but as the technology improves applications for near-point activities (reading and 
writing) may also involve changes to the current regime of rehabilitation strategies. 
Assuming the technology will ultimately be funded by third-party payers, if the 
rehabilitation strategies are highly specialized, it will be necessary to add new cer- 
tification requirements and a greater body of knowledge including the non-visual 
strategies and techniques for rehabilitation specialists serving this population. 

In conclusion, the use of prosthetic vision must be understood in the context of 
the client and his goals, lifestyle, and ability to adapt to change. Realistic expecta- 
tions and a high level of independence will enhance the chances of a positive out- 
come with prosthetic vision. Patients who are properly selected to participate in 
prosthetic vision rehabilitation need extensive education in regard to what pros- 
thetic vision intervention can and cannot do to assure they have realistic expecta- 
tions, which are vital to the success of rehabilitation. A realistic understanding of 
the potential of the technology, in combination with rehabilitation instruction, is 
one key to a successful outcome. 
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